String formats do not work

[adrtivv]

@sinclairzx81 String validation using the list of available formats does not work. Here's a reproducible stackblitz link.

Run node ./index.js and you can see that both the checks fail even though the string values conform to the specified formats.

[sinclairzx81]

@adrtivv Hi,

Unfortunately, TypeBox doesn't provide any string formats by default. But you can still register these formats manually via the FormatRegistry.

import { Type, FormatRegistry } from '@sinclair/typebox'
import { Value } from '@sinclair/typebox/value'
import { randomUUID } from 'node:crypto';

import { IsEmail, IsUuid } from './formats' // see below
FormatRegistry.Set('email', value => IsEmail(value))
FormatRegistry.Set('uuid', value => IsUuid(value))

const T = Type.Object({
  email: Type.String({ format: 'email' })
  uuid: Type.String({ format: 'uuid' })
})

const R = Value.Check(T, {
  email: 'test@gmail.com',
  uuid: randomUUID()
})

You can find a set of common formats (including email and uuid) in the examples/formats directory of this repo. These formats are ported from the ajv-formats project (specifically for Ajv compatibility) but modified to be made callable functions (enabling them to be registered in TypeBox). Unfortunately I won't be including these formats in the TypeBox package as standard (as a matter of etiquette and respect to the original authors), but they are offered in their example form as a reference.

Hope this helps!
S

[adrtivv]

Unfortunately, TypeBox doesn't provide any string formats by default.

Then why does typebox provide the option of passing in the format field in String types if it is useless by its own? Either it should be removed or it should be documented how to explicitly configure and enable it.

Anyway, did you mean say that I should manually define those functions isEmail and isUUID in my codebase for validation against those specific string formats? This seems like needless work, do you perhaps have some package in mind that can do this job for me?

[sinclairzx81]

Hiya,

Then why does typebox provide the option of passing in the format field in String types if it is useless by its own? Either it should be removed or it should be documented how to explicitly configure and enable it.

Keep in mind that TypeBox is primarily a Json Schema builder whose types are designed to be checked by any Json Schema compliant validator. Because of this, the format options are provided to be representative of the Json Schema specification formats (listed here), but TypeBox makes no assumptions as to if you're using Ajv, Hyperjump, TypeCompiler or other library. It also makes no assumptions with respect to the capabilities of each validator.

With respect to formats specifically, TypeBox avoids implementing them outright as there is some ambiguity as to what a format means across different systems, and some ambiguity to if a regular expression specified in C# or PHP is functionally equivalent to a similar format specified via ECMA 262 regular expression (or any other language). Because of this, TypeBox validation infrastructure requires the user (who was originally me) be specific about the format implementation.

Either it should be removed or it should be documented how to explicitly configure and enable it.

String format configuration is documented under https://github.com/sinclairzx81/typebox#formatregistry

The Role of Validators

From: https://json-schema.org/understanding-json-schema/reference/string#format

Implementations may provide validation for only a subset of the built-in formats or do partial validation for a given format. For example, some implementations may consider a string an email if it contains a @, while others might do additional checks for other aspects of a well formed email address.

The stance TypeBox (and Ajv) takes in this regard is that given the potential for formats to be interpreted differently based on application requirements, it doesn't necessarily make sense to provide a defacto default interpretation (but do appreciate it would be convenient if TypeBox did take a stronger stance and just picked a RFC default). As noted, Ajv takes a similar stance where configuration of formats is handled by way of the external package ajv-formats, in both libraries, the formats are opt in.

Options

Anyway, did you mean say that I should manually define those functions isEmail and isUUID in my codebase for validation against those specific string formats? This seems like needless work, do you perhaps have some package in mind that can do this job for me?

I do agree somewhat about the needless work, but unfortunately as of today, I don't think there exists a central package that implements a full suite of RFC compliant formats (other than ajv-formats). If you're just looking for relatively spec compliant format checkers, you can try integrate the ajv-formats package with FormatRegistry directly, alternatively you can use Ajv directly where things are a bit more integrated.

Note: if you do find a package that just implements formats, I would be very interested to know about it.

FormatRegistry + Ajv Formats

The following sets up the FormatRegistry using the get interface provided on default export for ajv-formats package. It's a little bit clunky, but it works.

import { Type, FormatRegistry } from '@sinclair/typebox'
import { Value } from '@sinclair/typebox/value'

// Use Formats from the Ajv Package
import Formats from 'ajv-formats'
FormatRegistry.Set('email', value => (Formats.get('email') as RegExp).test(value))
FormatRegistry.Set('uuid', value => (Formats.get('uuid') as RegExp).test(value))


const T = Type.String({ format: 'email' })

const R = Value.Check(T, 'user@domain.com') // true

Ajv + Ajv Formats

The following just uses the Ajv validator directly with Ajv Formats. The full setup can be found here

import { Type } from '@sinclair/typebox'
import Formats from 'ajv-formats'
import Ajv from 'ajv'

const ajv = Formats(new Ajv({}), ['email', 'uuid'])

const T = Type.String({ format: 'email' })

const R = ajv.validate(T, 'user@domain.com') // true

---

So, this is the current state of things with respect to string formats in TypeBox, and again I do appreciate it's a bit of an inconvenience. With all this said, the TypeBox project is open to community contributions to add fast, DoS free, RFC compliant formats to the library, but until such time, the recommendation is to configure the FormatRegistry with user defined formats inline with application requirements.

import { FormatRegistry } from '@sinclair/typebox'
import { IsEmail, IsUuid } from '@sinclair/typebox/formats' // one day (with community help)

Hope this brings some insight
Cheers
S

[adrtivv]

Thanks for the detailed explanation. Appreciate it.