Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ajv-validator/ajv
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v8.7.1
Choose a base ref
...
head repository: ajv-validator/ajv
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v8.8.0
Choose a head ref
  • 3 commits
  • 10 files changed
  • 2 contributors

Commits on Nov 13, 2021

  1. change exports to fix bundles (#1812)

    * change exports to fix bundles
    
    * use browserify for bundling
    
    * revert exports
    
    * use browserify in bundle script
    epoberezkin authored Nov 13, 2021
    Copy the full SHA
    f68ef8f View commit details
  2. option regExp to specify RegExp engine (e.g. re2) #1684

    commit 1835f3517ffb750ea4c75ce3ee8d9c262374e8f4
    Author: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    Date:   Sat Nov 13 18:04:08 2021 +0000
    
        simplify regExp option
    
    commit e7f1eb9
    Merge: 98f04d3 f68ef8f
    Author: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    Date:   Sat Nov 13 17:20:15 2021 +0000
    
        Merge branch 'master' into master
    
    commit 98f04d3
    Author: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    Date:   Sat Nov 13 17:20:04 2021 +0000
    
        Update docs/options.md
    
    commit 0ff99ed
    Merge: d9ea90c 8fccddb
    Author: Efe Barlas <43009963+efebarlas@users.noreply.github.com>
    Date:   Wed Nov 10 00:15:33 2021 -0500
    
        Merge branch 'master' into master
    
    commit d9ea90c
    Author: efebarlas <ebarlas@purdue.edu>
    Date:   Wed Nov 10 00:09:17 2021 -0500
    
        prettier:write to pass CI
    
        Signed-off-by: efebarlas <ebarlas@purdue.edu>
    
    commit b29cd91
    Merge: f50eb43 20089ed
    Author: efebarlas <ebarlas@purdue.edu>
    Date:   Tue Nov 9 21:54:45 2021 -0500
    
        Merge branch 'master' of github.com:efebarlas/ajv
        Tests added for code.regExp option
    
    commit f50eb43
    Author: efebarlas <ebarlas@purdue.edu>
    Date:   Tue Nov 9 21:54:28 2021 -0500
    
        Tests added
    
        Signed-off-by: efebarlas <ebarlas@purdue.edu>
    
    commit 20089ed
    Author: Efe Barlas <43009963+efebarlas@users.noreply.github.com>
    Date:   Tue Nov 9 21:53:34 2021 -0500
    
        Update options.md
    
    commit fd3e290
    Merge: 41dd4bc 6ef0c66
    Author: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    Date:   Sun Sep 12 19:07:28 2021 +0100
    
        Merge branch 'master' into master
    
    commit 41dd4bc
    Merge: 698f411 a9f38cd
    Author: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    Date:   Sun Sep 12 11:35:20 2021 +0100
    
        Merge branch 'master' into master
    
    commit 698f411
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Thu Aug 12 14:55:17 2021 -0400
    
        dev-dependency to node-re2 added
    
    commit a0720f8
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Thu Aug 12 14:43:39 2021 -0400
    
        re2 runtime lib + regExp code option added
    
    commit 1470c23
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Fri Jul 9 14:14:45 2021 -0400
    
        variable name changes
    
        Signed-off-by: Efe Barlas <ebarlas@purdue.edu>
    
    commit 8f7ca34
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Fri Jul 9 13:22:38 2021 -0400
    
        minor changes
    
        Signed-off-by: Efe Barlas <ebarlas@purdue.edu>
    
    commit 9791cce
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Fri Jul 9 13:20:47 2021 -0400
    
        remove comments
    
        Signed-off-by: Efe Barlas <ebarlas@purdue.edu>
    
    commit b07542d
    Author: Efe Barlas <ebarlas@purdue.edu>
    Date:   Fri Jul 9 11:28:29 2021 -0400
    
        added: RE2 Option with fallback
    
        Signed-off-by: Efe Barlas <ebarlas@purdue.edu>
    
    Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
    efebarlas and epoberezkin committed Nov 13, 2021

    Unverified

    The email in this signature doesn’t match the committer email.
    Copy the full SHA
    8ed855b View commit details
  3. 8.8.0

    epoberezkin committed Nov 13, 2021

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    62048a1 View commit details
Showing with 129 additions and 7 deletions.
  1. +7 −0 docs/options.md
  2. +9 −1 lib/core.ts
  3. +6 −0 lib/runtime/re2.ts
  4. +9 −0 lib/types/index.ts
  5. +7 −3 lib/vocabularies/code.ts
  6. +5 −2 package.json
  7. +1 −1 rollup.config.js
  8. +48 −0 scripts/bundle.js
  9. +33 −0 spec/issues/1683_re2_engine.spec.ts
  10. +4 −0 spec/issues/re2.ts
7 changes: 7 additions & 0 deletions docs/options.md
Original file line number Diff line number Diff line change
@@ -69,6 +69,7 @@ const defaultOptions = {
source: false,
process: undefined, // (code: string) => string
optimize: true,
regExp: RegExp
},
}
```
@@ -361,6 +362,12 @@ type CodeOptions = {
// Code snippet created with `_` tagged template literal that contains all format definitions,
// it can be the code of actual definitions or `require` call:
// _`require("./my-formats")`
regExp: RegExpEngine
// Pass non-standard RegExp engine to mitigate ReDoS, e.g. node-re2.
// During validation of a schema, code.regExp will be
// used to match strings against regular expressions.
// The supplied function must support the interface:
// regExp(regex, unicodeFlag).test(string) => boolean
}

type Source = {
10 changes: 9 additions & 1 deletion lib/core.ts
Original file line number Diff line number Diff line change
@@ -48,6 +48,7 @@ import type {
ErrorObject,
Format,
AddedFormat,
RegExpEngine,
} from "./types"
import type {JSONSchemaType} from "./types/json-schema"
import type {JTDSchemaType, SomeJTDSchemaType, JTDDataType} from "./types/jtd-schema"
@@ -62,6 +63,9 @@ import {eachItem} from "./compile/util"

import * as $dataRefSchema from "./refs/data.json"

const defaultRegExp: RegExpEngine = (str, flags) => new RegExp(str, flags)
defaultRegExp.code = "new RegExp"

const META_IGNORE_OPTIONS: (keyof Options)[] = ["removeAdditional", "useDefaults", "coerceTypes"]
const EXT_SCOPE_NAMES = new Set([
"validate",
@@ -141,9 +145,11 @@ export interface CodeOptions {
formats?: Code // code to require (or construct) map of available formats - for standalone code
source?: boolean
process?: (code: string, schema?: SchemaEnv) => string
regExp?: RegExpEngine
}

interface InstanceCodeOptions extends CodeOptions {
regExp: RegExpEngine
optimize: number
}

@@ -231,13 +237,14 @@ function requiredOptions(o: Options): RequiredInstanceOptions {
const s = o.strict
const _optz = o.code?.optimize
const optimize = _optz === true || _optz === undefined ? 1 : _optz || 0
const regExp = o.code?.regExp ?? defaultRegExp
return {
strictSchema: o.strictSchema ?? s ?? true,
strictNumbers: o.strictNumbers ?? s ?? true,
strictTypes: o.strictTypes ?? s ?? "log",
strictTuples: o.strictTuples ?? s ?? "log",
strictRequired: o.strictRequired ?? s ?? false,
code: o.code ? {...o.code, optimize} : {optimize},
code: o.code ? {...o.code, optimize, regExp} : {optimize, regExp},
loopRequired: o.loopRequired ?? MAX_EXPRESSION,
loopEnum: o.loopEnum ?? MAX_EXPRESSION,
meta: o.meta ?? true,
@@ -279,6 +286,7 @@ export default class Ajv {
constructor(opts: Options = {}) {
opts = this.opts = {...opts, ...requiredOptions(opts)}
const {es5, lines} = this.opts.code

this.scope = new ValueScope({scope: {}, prefixes: EXT_SCOPE_NAMES, es5, lines})
this.logger = getLogger(opts.logger)
const formatOpt = opts.validateFormats
6 changes: 6 additions & 0 deletions lib/runtime/re2.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import * as re2 from "re2"

type Re2 = typeof re2 & {code: string}
;(re2 as Re2).code = 'require("ajv/dist/runtime/re2").default'

export default re2 as Re2
9 changes: 9 additions & 0 deletions lib/types/index.ts
Original file line number Diff line number Diff line change
@@ -222,3 +222,12 @@ export type AddedFormat =
| AsyncFormatDefinition<number>

export type Format = AddedFormat | string

export interface RegExpEngine {
(pattern: string, u: string): RegExpLike
code: string
}

export interface RegExpLike {
test: (s: string) => boolean
}
10 changes: 7 additions & 3 deletions lib/vocabularies/code.ts
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ import type {KeywordCxt} from "../compile/validate"
import {CodeGen, _, and, or, not, nil, strConcat, getProperty, Code, Name} from "../compile/codegen"
import {alwaysValidSchema, Type} from "../compile/util"
import N from "../compile/names"

import {useFunc} from "../compile/util"
export function checkReportMissingProp(cxt: KeywordCxt, prop: string): void {
const {gen, data, it} = cxt
gen.if(noPropertyInData(gen, data, prop, it.opts.ownProperties), () => {
@@ -90,12 +90,16 @@ export function callValidateCode(
return context !== nil ? _`${func}.call(${context}, ${args})` : _`${func}(${args})`
}

const newRegExp = _`new RegExp`

export function usePattern({gen, it: {opts}}: KeywordCxt, pattern: string): Name {
const u = opts.unicodeRegExp ? "u" : ""
const {regExp} = opts.code

return gen.scopeValue("pattern", {
key: pattern,
ref: new RegExp(pattern, u),
code: _`new RegExp(${pattern}, ${u})`,
ref: regExp(pattern, u),
code: _`${regExp.code === "new RegExp" ? newRegExp : useFunc(gen, regExp)}(${pattern}, ${u})`,
})
}

7 changes: 5 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ajv",
"version": "8.7.1",
"version": "8.8.0",
"description": "Another JSON Schema Validator",
"main": "dist/ajv.js",
"types": "dist/ajv.d.ts",
@@ -17,7 +17,8 @@
"test-codegen": "nyc cross-env TS_NODE_PROJECT=spec/tsconfig.json mocha -r ts-node/register 'spec/codegen.spec.ts' -R spec",
"test-debug": "npm run test-spec -- --inspect-brk",
"test-cov": "nyc npm run test-spec",
"bundle": "rm -rf bundle && rollup -c",
"rollup": "rm -rf bundle && rollup -c",
"bundle": "rm -rf bundle && node ./scripts/bundle.js ajv ajv7 ajv7 && node ./scripts/bundle.js 2019 ajv2019 ajv2019 && node ./scripts/bundle.js 2020 ajv2020 ajv2020 && node ./scripts/bundle.js jtd ajvJTD ajvJTD",
"build": "rm -rf dist && tsc && cp -r lib/refs dist && rm dist/refs/json-schema-2019-09/index.ts && rm dist/refs/json-schema-2020-12/index.ts && rm dist/refs/jtd-schema.ts",
"json-tests": "rm -rf spec/_json/*.js && node scripts/jsontests",
"test-karma": "karma start",
@@ -75,6 +76,7 @@
"@typescript-eslint/eslint-plugin": "^3.8.0",
"@typescript-eslint/parser": "^3.8.0",
"ajv-formats": "^3.0.0-rc.0",
"browserify": "^17.0.0",
"chai": "^4.0.1",
"cross-env": "^7.0.2",
"dayjs": "^1.10.4",
@@ -95,6 +97,7 @@
"node-fetch": "^3.0.0",
"nyc": "^15.0.0",
"prettier": "^2.3.1",
"re2": "^1.16.0",
"rollup": "^2.44.0",
"rollup-plugin-terser": "^7.0.2",
"ts-node": "^10.0.0",
2 changes: 1 addition & 1 deletion rollup.config.js
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@ function createBundleConfig(sourceFile, outFile, globalName) {
}

export default [
createBundleConfig("ajv", "ajv7", "av7"),
createBundleConfig("ajv", "ajv7", "ajv7"),
createBundleConfig("2019", "ajv2019", "ajv2019"),
createBundleConfig("2020", "ajv2020", "ajv2020"),
createBundleConfig("jtd", "ajvJTD", "ajvJTD"),
48 changes: 48 additions & 0 deletions scripts/bundle.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"use strict"

const fs = require("fs")
const path = require("path")
const browserify = require("browserify")
const {minify} = require("terser")

const [sourceFile, outFile, globalName] = process.argv.slice(2)

const json = require(path.join(__dirname, "..", "package.json"))
const bundleDir = path.join(__dirname, "..", "bundle")
if (!fs.existsSync(bundleDir)) fs.mkdirSync(bundleDir)

browserify({standalone: globalName})
.require(path.join(__dirname, "../dist", sourceFile), {expose: sourceFile})
.bundle(saveAndMinify)

async function saveAndMinify(err, buf) {
if (err) {
console.error("browserify error:", err)
process.exit(1)
}

const bundlePath = path.join(bundleDir, outFile)
const opts = {
ecma: 2018,
warnings: true,
compress: {
pure_getters: true,
keep_infinity: true,
unsafe_methods: true,
},
format: {
preamble: `/* ${json.name} ${json.version} (${globalName}): ${json.description} */`,
},
sourceMap: {
filename: outFile + ".min.js",
url: outFile + ".min.js.map",
},
}

const result = await minify(buf.toString(), opts)

fs.writeFileSync(bundlePath + ".bundle.js", buf)
fs.writeFileSync(bundlePath + ".min.js", result.code)
fs.writeFileSync(bundlePath + ".min.js.map", result.map)
if (result.warnings) result.warnings.forEach((msg) => console.warn("terser.minify warning:", msg))
}
33 changes: 33 additions & 0 deletions spec/issues/1683_re2_engine.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import getAjvAllInstances from "../ajv_all_instances"
import {withStandalone} from "../ajv_standalone"
import {_} from "../../dist/compile/codegen/code"
import jsonSchemaTest = require("json-schema-test")
import options from "../ajv_options"
import {afterError, afterEach} from "../after_test"
import chai from "../chai"
import re2 from "../../dist/runtime/re2"
import re2tests from "./re2"

const instances = getAjvAllInstances(options, {
$data: true,
formats: {allowedUnknown: true},
strictTypes: false,
strictTuples: false,
})

instances.forEach((ajv) => {
ajv.opts.code.source = true
ajv.opts.code.formats = _`{allowedUnknown: true}`
ajv.opts.code.regExp = re2
})

jsonSchemaTest(withStandalone(instances), {
description: "Test with re2 RegExp engine with " + instances.length + " ajv instances",
suites: {"regular expressions": re2tests},
assert: chai.assert,
afterError,
afterEach,
cwd: __dirname,
hideFolder: "extras/",
timeout: 90000,
})
4 changes: 4 additions & 0 deletions spec/issues/re2.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export default [
{name: "$data/format", test: require("../extras/$data/format.json")},
{name: "$data/pattern", test: require("../extras/$data/pattern.json")},
]