User defined formats with standalone validation code

[Nantris]

The error we're seeing is:

Error: CodeGen: "code" for formats0 not defined

Oddly this error does not occur if we use Ajv in our app, it only occurs if we try to make a standalone validator script.

What version of Ajv are you using? Does the issue happen if you use the latest version? 7.1.1 (latest)

Ajv options object

const ajv = new Ajv({
  code: {
    source: true, // this option is required to generate standalone code
  },
  removeAdditional: true,
});

JSON Schema

const demoSchema = {
  $id: 'demoSchema',
  additionalProperties: false,
  properties: {
    test: hexCodeValidator,
  },
};

Sample data

N/A - bug occurs during compilation of standalone Ajv.

Your code

Repro

Are you going to resolve the issue?

I cannot resolve the issue.

[epoberezkin]

Custom format functions support with standalone code is limited.

If simply using regular expression (not a function) works for your case - it works for the example, but maybe you have a more complex function, - then passing regular expression directly to addFormat would make it work.

If you do need a function then there is an instance option (see Options in api.md) to which you need to pass a “code snippet” that would result into an object with all formats - see ajv-formats here for the example how it can be done:

https://github.com/ajv-validator/ajv-formats/blob/master/src/index.ts#L55

An obvious improvement is to allow defining custom formats individually and pass “code snippets” for standalone code in their definitions.

It’s not well documented,

[Nantris]

@epoberezkin thanks for your speedy reply!

I think regex alone would work for us but I'm not clear on the syntax in that case. I took a look at the docs and I tried the following:

  ajv.addFormat('hex-code', hexCodeRegex);`

and

  ajv.addFormat('hex-code', {
    validate: hexCodeRegex,
  });

but both result in the same error. Any thoughts?

[epoberezkin]

I expected this to work: ajv.addFormat('hex-code', hexCodeRegex);

(assuming hexCodeRegex is RegExp instance)

need to have a look why it’s failing

[Nantris]

I made a new repro but it throws an error about unexpected token instead of producing the bug.

The JS validates though... Seems like a Runkit thing?

https://runkit.com/slapbox/603d6ad7a58cfe001a9e648e

[Nantris]

@epoberezkin any thoughts on this? Either on the original issue at hand or why the repro won't run?

[Nantris]

@epoberezkin I moved the code to CodeSandbox where it doesn't have that error. I also upgraded to 8.0.1. The resulting error now is: CodeGen: "code" for formats0 not defined

https://codesandbox.io/s/relaxed-bhaskara-5bisx?file=/src/index.js

Any thoughts?

[Nantris]

I just updated to 8.0.1 in our actual project to try in a more well known environment and it generates the same error, CodeGen: "code" for formats0 not defined.

We're hoping to push a new release in a couple weeks and this issue is a blocker for us. I'd love to be of more help resolving it but I think all I can do is report findings unfortunately. If there's anything I can do to help, let me know!

[epoberezkin]

The required minimal change is to support formats defined as regular expressions without the need to set code.formats option - the change would be in /lib/vocabularies/format/format.ts - the PR is welcome.

More complex change that I wasn’t planning at all would be to define code snippet per each format, not only on the instance level, to support formats defined with “validate” function. That can be also done for user defined keywords (those that use “validate” function) - this is not in scope of this issue.

The workaround “should be” quite straightforward, as I suggested above - you need to put all your formats in a separate file and pass code snippet with require of this file to Ajv instance via code.formats option - similar to how ajv-formats is doing it - see the link above.

[Nantris]

The required minimal change is to support formats defined as regular expressions without the need to set code.formats option - the change would be in /lib/vocabularies/format/format.ts - the PR is welcome.

I'm not following when ajv-formats would be needed for RegExp support versus what ajv should handle out of the box?

Just linking to the file here for convenience: https://github.com/ajv-validator/ajv/blob/master/lib/vocabularies/format/format.ts

[Nantris]

Oops, submitted that early and accidentally closed.

I'm not really able to follow what would need to be done for a PR solution unfortunately, and I'm also not the most proficient in TypeScript which makes it a bit harder still to jump right into the code. I'll peruse the docs a bit and see if I can get a more firm understanding of the code.formats option and related functionality.

[epoberezkin]

No worries.

I'm not following when ajv-formats would be needed for RegExp support versus what ajv should handle out of the box?

Ajv should handle regexps out of the box.

code.formats option

As an example you could have a file with all your formats, say formats.js:

module.exports = {
  "hex-code": /.*/,  // or anything you need
  // more formats you need
  // and you can mix-in ajv-formats here as well, if needed, instead of calling ajv-formats(ajv)
}

Then when you initialise ajv you would:

const Ajv = require("ajv")
const {_} = Ajv
const ajv = new Ajv({
  formats: require("./formats.js"),
  code: {formats: _`require(${path_to_your_formats_module}/formats.js)`}
})

This path should be relative to the location of your compiled validation code module - so it would depend on where you place it - it most cases it would not be a variable.

Hope it makes sense

[epoberezkin]

this is not well documented really - needs to be added to the docs

[Nantris]

Thanks a ton for taking the time to clarify that! I think I'm at least somewhat following now.

I doubt I ever would have wrapped my head around how to fix this without that walkthrough. Thanks to your help though, I did manage to fix our issue with using hexcode regex!

Unfortunately I think I'm still a ways from grokking enough to attempt a non-mangled PR - though at least I'm not completely lost when I look over format.ts now.

But if you can point me to where in the docs the code.formats stuff should be mentioned, maybe I can submit a PR to address the documentation at least.

Thanks again very much for your help @epoberezkin!

[epoberezkin]

Yes - adding to docs would be great. I suggest amending code example in https://ajv.js.org/guide/formats.html#formats-and-standalone-validation-code (I actually forgot there was something there:) and reference it from options: https://ajv.js.org/options.html#code

[epoberezkin]

@slapbox RegExp format support in standalone code added in v8.0.2

Still worth amending the docs, for the formats defined as functions.

[Nantris]

So the new code example would be like this, right? The current code example doesn't include source even though it's discussing standalone validation, so I've added that and some comments. Let me know if you think the comments might be better mentioned elsewhere instead of the comments.

const { default: Ajv, _ } = require('ajv');

const ajv = new Ajv({
  formats: require('./my_formats'),
  code: { 
    source: true, // Generate standalone validation code
    formats: _`require("./my_formats")` },  //  This path must be relative to the compiled standalone validation code at runtime
});

---

And then on https://ajv.js.org/options.html#code just mention to see the page https://ajv.js.org/guide/formats.html#formats-and-standalone-validation-code for more information on how to use formats? Or link specifically to the sub-heading? https://ajv.js.org/guide/formats.html#formats-and-standalone-validation-code

Could you point me to an example of a non-RegExp custom format? Since our use case is now supported without the extra steps, it wouldn't make for a great example. Is it now basically just formats which provide custom validate functions?

[epoberezkin]

So the new code example would be like this, right?

yes

Or link specifically to the sub-heading? https://ajv.js.org/guide/formats.html#formats-and-standalone-validation-code

I think so - to point to the example there (that would be extended).

Is it now basically just formats which provide custom validate functions?

Yes. Some string formats can be easier to write as a function than regexp.