Parsing JSON schema into TypeBox

[Tydax]

Hello there!

First of all, I want to applaud and thank you for your fantastic work for this library, it is a great concept and I love its overall readability.

I am currently using this for a project where I need to generalise data structures stored into persistence while still needing validation and type inference. TypeBox really assisted me in writing JSON schemas directly into my codebase to re-use into these data structures.

However, I am encountering a small pain point, where I want to be able to use these schemas stored in persistence. Exporting is no issue, as I can utilise Type.Strict() to basically “convert” the TypeBox schema into a regular JSON schema; but I did not find a way to “import” a JSON schema into TypeBox (which I am assuming would mostly consist of adding the [Kind] and [Modifier] properties to the schema object). Since these custom properties are missing from the schemas once exported, it means that they cannot be used at all with TypeBox.
For the time being, to validate data against my schemas, I rely on ajv since it is designed for that purpose, but I wish I could use more of the TypeBox features for these schemas like value generations.

I am not sure if this feature exists, or maybe I missed it during my research on the documentation and discussions, but does TypeBox offer such a feature? And if not, would you consider adding it?

Many thanks!

[sinclairzx81]

@Tydax Hi, thanks for the kind words on TypeBox! :)

Um, at this stage, there are no immediate plans to support loading / parsing schemas from disk, but there are some considerations going into removing the [Kind] and [Modifier] symbols (which I'm not sure is possible, but if achieved, might make inroads into parsing structures from disk a bit more feasible). The problem currently is that once the schematic is serialized via Strict (or simply JSON.stringify()), associative static type information is lost and it's difficult to parse them back to their original type (as these Symbols encode all information related to the static type, which also has implications as to how the types compose at runtime)

The classic example of where differentiating schematics becomes a problem is between any and unknown type representations. Where both of these types share a common representation.

const A = Type.Strict(Type.Any())     // const A = { } ────┐
                                      //                   ├─ Should parse as Any or Unknown?
const B = Type.Strict(Type.Unknown()) // const B = { } ────┘

Attempts have been made to keep TB using a strict subset of the JSON Schema specification that could make deserialization (and parsing to the original type) somewhat possible (through some form of schema pattern matching), but cases like the above (as well as String patterns, Record keys, and several other constructs do make loading the intended type prohibitively difficult. However, in theory, loading a Any as an Unknown, or a TemplateLiteral as a String with a pattern may not be a huge downside as the schema representation means basically both types assert the same way at runtime.

In saying this, I would like to see the following achieved in some capacity, but possibly as an auxiliary library.

import { TSchema, TypeGuard } from '@sinclair/typebox'

const T: TSchema = load('./schemas/A.json') // introspect the `A.json` schema and load as `TSchema`

const R = TypeGuard.TSchema(T)              // const R = true (if loading was successful)

In this scenario, the load() function would make a "best guess" at what the TSchema type should be by introspecting the structures found in the A.json schema. The TypeGuard would then be used to assert the type loaded as expected. Work to carry out this form of loading could be written as one of the TB example projects in the short term. If there was consensus reached on how to resolve for some of these more difficult cases, it could be promoted to standard (and as a compliment inverse of Strict())

For now though, loading from disk (or other form of persistence) is somewhat deferred indefinitely until some of these cases are solved for. I'm still not sure of the best approach to take here (this question has been asked a few times over the years), but community experimentation around this aspect would help greatly define how this functionality should work and help find where the edge cases are.

Hope this brings a bit of insight and background into this aspect of TB! :)
Happy to discuss more.
All the best
S

[Tydax]

Hey, thanks for the reply! I think your suggested approaches sound great. I may look into experimenting with TypeBox, and see what could be achieved if I find some time. I will keep you updated if I do :)

All the best!