url: use ada::url_aggregator for parsing urls #47339

anonrig · 2023-03-31T13:08:01Z

This pull request:

updates ada and replaces ada::url with ada::url_aggregator in correct places
removes ICU requirement from URL
improves the performance of URL parsing significantly

The release notes for Ada v2.0 can be found from https://www.yagiz.co/announcing-ada-url-parser-v2-0/

Added the notable-change label due to removing the requirement for ICU for proper hostname parsing.

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1314/

url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='auth' withBase='false'              ***    106.73 %      ±19.04% ±25.55% ±33.70%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='auth' withBase='true'               ***    111.98 %      ±18.10% ±24.30% ±32.05%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='dot' withBase='false'               ***    112.52 %      ±16.28% ±21.83% ±28.73%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='dot' withBase='true'                ***     91.22 %      ±17.19% ±23.05% ±30.35%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='file' withBase='false'              ***     31.72 %      ±12.53% ±16.68% ±21.72%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='file' withBase='true'               ***     31.05 %      ±12.37% ±16.46% ±21.43%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='idn' withBase='false'               ***     83.54 %      ±15.31% ±20.49% ±26.90%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='idn' withBase='true'                ***     81.76 %      ±16.08% ±21.57% ±28.44%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='javascript' withBase='false'        ***    137.30 %      ±18.20% ±24.43% ±32.23%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='javascript' withBase='true'         ***    111.40 %      ±16.83% ±22.53% ±29.58%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='long' withBase='false'              ***     25.11 %       ±8.80% ±11.73% ±15.28%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='long' withBase='true'               ***     23.16 %       ±8.63% ±11.52% ±15.06%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='percent' withBase='false'           ***     41.12 %      ±14.86% ±19.79% ±25.79%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='percent' withBase='true'            ***     37.21 %      ±12.93% ±17.23% ±22.48%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='short' withBase='false'             ***    123.39 %      ±18.93% ±25.36% ±33.34%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='short' withBase='true'              ***    103.11 %      ±17.88% ±23.97% ±31.54%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='wpt' withBase='false'               ***     35.04 %       ±2.25%  ±3.02%  ±3.98%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='wpt' withBase='true'                ***      9.76 %       ±4.21%  ±5.62%  ±7.36%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='ws' withBase='false'                ***    133.03 %      ±19.31% ±25.91% ±34.16%
url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='ws' withBase='true'                 ***    113.69 %      ±18.69% ±25.08% ±33.08%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 40 comparisons, you can thus
expect the following amount of false-positive results:
  2.00 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.40 false positives, when considering a   1% risk acceptance (**, ***),
  0.04 false positives, when considering a 0.1% risk acceptance (***)

cc @lemire @miguelteixeiraa

nodejs-github-bot · 2023-03-31T13:08:06Z

Review requested:

@nodejs/gyp
@nodejs/startup
@nodejs/url

tniessen

Is this marked as notable because it removes the need for ICU to use URL?

doc/api/url.md

nodejs-github-bot · 2023-03-31T13:51:21Z

CI: https://ci.nodejs.org/job/node-test-pull-request/50748/

anonrig · 2023-03-31T14:59:29Z

@joyeecheung parallel.test-shadow-realm-gc is failing. Any idea why?

cc @legendecas It seems its related to: #46809

lib/internal/url.js

src/node_url.cc

lib/internal/url.js

joyeecheung · 2023-03-31T16:05:13Z

parallel.test-shadow-realm-gc is failing. Any idea why?

My guess is by adding a new binding data to the context this tips the memory needed by creating a new shadow realm over what's assumed by the test. You can try logging out process.memoryUsage() to see how much more memory is used after the change. If that's the case you can try bumping the --max-old-space-size in that test a bit (but not too much to let a leak get away)

anonrig · 2023-03-31T16:29:18Z

My guess is by adding a new binding data to the context this tips the memory needed by creating a new shadow realm over what's assumed by the test. You can try logging out process.memoryUsage() to see how much more memory is used after the change. If that's the case you can try bumping the --max-old-space-size in that test a bit (but not too much to let a leak get away)

If I understood the concept of shadow realm, this particular test is creating 1000 UInt32Array (each realm creates 1 due to binding values in node_url.cc), since every realm is isolated. I'm not quite sure what the correct value is for --max-old-space-size and how to optimize the value for the concerns you've stated. For example even --max-old-space-size=500 does not fix the problem for me.

Updating to --max-old-space-size=1000 fixed, but this particular test takes 14 seconds at the moment.

➜  node git:(update-ada-v2) ✗ time tools/test.py test/parallel/test-shadow-realm-gc.js
[00:08|% 100|+   1|-   0]: Done

All tests passed.
tools/test.py test/parallel/test-shadow-realm-gc.js  14.63s user 0.78s system 177% cpu 8.683 total

bricss · 2023-03-31T19:40:23Z

Please somebody tell me that the next step 🥾 would be implementation of URLPattern API? 🧐

PR-URL: nodejs#47339 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

PR-URL: #47339 Backport-PR-URL: #48345 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 deps: * update ada to 2.0.0 (Node.js GitHub Bot) #47339 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * drop ICU requirement for parsing hostnames (Yagiz Nizipli) #47339 * use ada::url_aggregator for parsing urls (Yagiz Nizipli) #47339 * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 * use WebIDL converters in WebCryptoAPI (Filip Skokan) #46067 deps: * update ada to 2.0.0 (Node.js GitHub Bot) #47339 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * drop ICU requirement for parsing hostnames (Yagiz Nizipli) #47339 * use ada::url_aggregator for parsing urls (Yagiz Nizipli) #47339 * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in #47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in #46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in #47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in #46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 * (SEMVER-MINOR) implement AbortSignal.any() (Chemi Atlow) #47821 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

…atch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in nodejs#47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in nodejs#46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) nodejs#47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) nodejs#46973 doc: * add ovflowd to collaborators (Claudio Wunder) nodejs#47844 * add KhafraDev to collaborators (Matthew Aitken) nodejs#47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) nodejs#47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) nodejs#41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) nodejs#46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) nodejs#47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) nodejs#47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) nodejs#47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) nodejs#46190 * (SEMVER-MINOR) implement AbortSignal.any() (Chemi Atlow) nodejs#47821 module: * change default resolver to not throw on unknown scheme (Gil Tayar) nodejs#47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) nodejs#48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) nodejs#46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) nodejs#47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (nodejs#46929) (Robert Nagy) nodejs#46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) nodejs#48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) nodejs#47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) nodejs#47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) nodejs#47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) nodejs#47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) nodejs#48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) nodejs#47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) nodejs#47286 PR-URL: nodejs#48694

…atch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0

* chore: bump node in DEPS to v18.17.0 * chore: update build_modify_js2c_py_to_allow_injection_of_original-fs_and_custom_embedder_js.patch Xref: nodejs/node#46930 manually sync patch to minor upstream code shear * chore: update build_ensure_native_module_compilation_fails_if_not_using_a_new.patch Xref: nodejs/node#48248 manually sync patch to minor upstream code shear * chore: update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch Xref: nodejs/node#47824 chore: upstream func throwIfUnsupportedURLProtocol() has been removed, so no need to patch it * chore: update api_pass_oomdetails_to_oomerrorcallback.patch Xref: nodejs/node#47695 manually sync patch to minor upstream code shear * chore: remove fix_prevent_changing_functiontemplateinfo_after_publish.patch Xref: nodejs/node#46979 (upstreamed patch) Xref: https://chromium-review.googlesource.com/c/v8/v8/+/2718147 (related) * chore: update fix_adapt_debugger_tests_for_upstream_v8_changes.patch Xref: nodejs/node#47274 manually sync patch to minor upstream code shear some tests moved from sequential to parallel * chore: remove fix_libc_buffer_overflow_in_string_view_ctor.patch Xref: fix_libc_buffer_overflow_in_string_view_ctor.patch patch is no longer needed due to upstream bump to ada 2.2.0 * chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.patch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0 * chore: rebuild filenames.json several files removed/added/changed upstream * chore: update build_add_gn_build_files.patch upstream dep histogram 0.11.7 moved its include path from src/ to include/ Xref: nodejs/node#47742 * chore: update fix_crypto_tests_to_run_with_bssl.patch Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: ftbfs in node dep ada * fix: ftbfs in node dep uvwasi * chore: rebuild patches * chore: update fix_handle_boringssl_and_openssl_incompatibilities.patch Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: work around Node 18 isURL() regression * chore: sort script/node-disabled-tests.json alphabetically * test: add parallel/test-snapshot-argv1 to disabled list test: add parallel/test-snapshot-namespaced-builtin to disabled list We don't support that type of snapshotting at the moment. * chore: disable flaky node test parallel/test-dgram-send-cb-quelches-error fails upstream in v18.x on my box as well * ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set * fixup! fix: work around Node 18 isURL() regression fix: infinite loop regression * fixup! fix: work around Node 18 isURL() regression * chore: patch fixtures/errors/force_colors.snapshot The line numbers in the stacktrace from our v8 build don't match what Node's tests are expecting, so update the stacktrace to match our build. The specific numbers probably aren't t needed for the force_colors test, which is trying to see whether or not the lines are greyed out. One option is to upstream a test change to stop hardcoding the stacktrace. * fixup! fix: work around Node 18 isURL() regression fix; pull in upstream bugfix * fixup! ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set chore: do not inject ELECTRON_RUN_AS_NODE in test-assert-colors.js * chore: disable flaky node test parallel/test-debugger-random-port-with-inspect-port --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in nodejs#47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in nodejs#46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) nodejs#47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) nodejs#46973 doc: * add ovflowd to collaborators (Claudio Wunder) nodejs#47844 * add KhafraDev to collaborators (Matthew Aitken) nodejs#47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) nodejs#47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) nodejs#41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) nodejs#46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) nodejs#47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) nodejs#47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) nodejs#47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) nodejs#46190 * (SEMVER-MINOR) implement AbortSignal.any() (Chemi Atlow) nodejs#47821 module: * change default resolver to not throw on unknown scheme (Gil Tayar) nodejs#47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) nodejs#48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) nodejs#46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) nodejs#47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (nodejs#46929) (Robert Nagy) nodejs#46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) nodejs#48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) nodejs#47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) nodejs#47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) nodejs#47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) nodejs#47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) nodejs#48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) nodejs#47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) nodejs#47286 PR-URL: nodejs#48694

The built-in URL class dropped support for zone identifiers in IPv6 address literals in Node.js 20. Calling the URL constructor with a URL containing a zone identifier causes ERR_INVALID_URL to be thrown. This is likely a result of switching to version 2.0 of the Ada URL parser in <nodejs/node#47339>. The behavior aligns with how [IPv6 address is defined in the WHATWG URL Standard](https://url.spec.whatwg.org/#concept-ipv6), which notes that > Support for <zone_id> is intentionally omitted. As explained in the issue tracker: https://www.w3.org/Bugs/Public/show_bug.cgi?id=27234#c2 whatwg/url#392 Skip this test, since this URL format is not supported. If it's necessary to support SCP-like git URLs with zone identifiers, we'll need to roll our own support. Signed-off-by: Kevin Locke <kevin@kevinlocke.name>

* chore: bump node in DEPS to v18.17.0 * chore: update build_modify_js2c_py_to_allow_injection_of_original-fs_and_custom_embedder_js.patch Xref: nodejs/node#46930 manually sync patch to minor upstream code shear * chore: update build_ensure_native_module_compilation_fails_if_not_using_a_new.patch Xref: nodejs/node#48248 manually sync patch to minor upstream code shear * chore: update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch Xref: nodejs/node#47824 chore: upstream func throwIfUnsupportedURLProtocol() has been removed, so no need to patch it * chore: update api_pass_oomdetails_to_oomerrorcallback.patch Xref: nodejs/node#47695 manually sync patch to minor upstream code shear * chore: remove fix_prevent_changing_functiontemplateinfo_after_publish.patch Xref: nodejs/node#46979 (upstreamed patch) Xref: https://chromium-review.googlesource.com/c/v8/v8/+/2718147 (related) * chore: update fix_adapt_debugger_tests_for_upstream_v8_changes.patch Xref: nodejs/node#47274 manually sync patch to minor upstream code shear some tests moved from sequential to parallel * chore: remove fix_libc_buffer_overflow_in_string_view_ctor.patch Xref: fix_libc_buffer_overflow_in_string_view_ctor.patch patch is no longer needed due to upstream bump to ada 2.2.0 * chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.patch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0 * chore: rebuild filenames.json several files removed/added/changed upstream * chore: update build_add_gn_build_files.patch upstream dep histogram 0.11.7 moved its include path from src/ to include/ Xref: nodejs/node#47742 * chore: update fix_crypto_tests_to_run_with_bssl.patch Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: ftbfs in node dep ada * fix: ftbfs in node dep uvwasi * chore: rebuild patches * chore: update fix_handle_boringssl_and_openssl_incompatibilities.patch Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: work around Node 18 isURL() regression * chore: sort script/node-disabled-tests.json alphabetically * test: add parallel/test-snapshot-argv1 to disabled list test: add parallel/test-snapshot-namespaced-builtin to disabled list We don't support that type of snapshotting at the moment. * chore: disable flaky node test parallel/test-dgram-send-cb-quelches-error fails upstream in v18.x on my box as well * ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set * fixup! fix: work around Node 18 isURL() regression fix: infinite loop regression * fixup! fix: work around Node 18 isURL() regression * chore: patch fixtures/errors/force_colors.snapshot The line numbers in the stacktrace from our v8 build don't match what Node's tests are expecting, so update the stacktrace to match our build. The specific numbers probably aren't t needed for the force_colors test, which is trying to see whether or not the lines are greyed out. One option is to upstream a test change to stop hardcoding the stacktrace. * fixup! fix: work around Node 18 isURL() regression fix; pull in upstream bugfix * fixup! ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set chore: do not inject ELECTRON_RUN_AS_NODE in test-assert-colors.js * chore: disable flaky node test parallel/test-debugger-random-port-with-inspect-port --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>

anonrig added url Issues and PRs related to the legacy built-in url module. notable-change PRs with changes that should be highlighted in changelogs. whatwg-url Issues and PRs related to the WHATWG URL implementation. labels Mar 31, 2023

This comment was marked as resolved.

Sign in to view

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Mar 31, 2023

anonrig force-pushed the update-ada-v2 branch 2 times, most recently from 05408df to d0da46f Compare March 31, 2023 13:12

tniessen reviewed Mar 31, 2023

View reviewed changes

doc/api/url.md Show resolved Hide resolved

anonrig mentioned this pull request Mar 31, 2023

deps: update ada to 2.0.0 #47332

Closed

anonrig force-pushed the update-ada-v2 branch from d0da46f to da589fd Compare March 31, 2023 13:19

anonrig added commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. performance Issues and PRs related to the performance of Node.js. request-ci Add this label to start a Jenkins CI on a PR. labels Mar 31, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 31, 2023

lemire reviewed Mar 31, 2023

View reviewed changes

lib/internal/url.js Outdated Show resolved Hide resolved

lemire reviewed Mar 31, 2023

View reviewed changes

lib/internal/url.js Outdated Show resolved Hide resolved

lemire reviewed Mar 31, 2023

View reviewed changes

lib/internal/url.js Outdated Show resolved Hide resolved

lemire reviewed Mar 31, 2023

View reviewed changes

src/node_url.cc Show resolved Hide resolved

joyeecheung reviewed Mar 31, 2023

View reviewed changes

lib/internal/url.js Show resolved Hide resolved

anonrig force-pushed the update-ada-v2 branch from da589fd to ad5684d Compare March 31, 2023 16:24

anonrig force-pushed the update-ada-v2 branch 3 times, most recently from 2cd305b to 2a2ed23 Compare March 31, 2023 19:18

anonrig pushed a commit to anonrig/node that referenced this pull request Jul 9, 2023

deps: update ada to 2.0.0

f9259fd

PR-URL: nodejs#47339 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

anonrig added a commit to anonrig/node that referenced this pull request Jul 9, 2023

url: use ada::url_aggregator for parsing urls

7728eee

PR-URL: nodejs#47339 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

anonrig added a commit to anonrig/node that referenced this pull request Jul 9, 2023

url: drop ICU requirement for parsing hostnames

d8c2a66

PR-URL: nodejs#47339 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

danielleadams pushed a commit that referenced this pull request Jul 12, 2023

deps: update ada to 2.0.0

96c50ba

PR-URL: #47339 Backport-PR-URL: #48345 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

danielleadams pushed a commit that referenced this pull request Jul 12, 2023

url: use ada::url_aggregator for parsing urls

b395b16

PR-URL: #47339 Backport-PR-URL: #48345 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

danielleadams pushed a commit that referenced this pull request Jul 12, 2023

url: drop ICU requirement for parsing hostnames

0dc485e

PR-URL: #47339 Backport-PR-URL: #48345 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

danielleadams mentioned this pull request Jul 12, 2023

v18.17.0 release proposal #48694

Merged

danielleadams added backported-to-v18.x PRs backported to the v18.x-staging branch. and removed backport-open-v18.x Indicate that the PR has an open backport. labels Jul 12, 2023

ckerr added a commit to electron/electron that referenced this pull request Jul 19, 2023

chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.p…

13a8181

…atch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0

ckerr added a commit to electron/electron that referenced this pull request Jul 25, 2023

chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.p…

5d03e93

…atch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0

styfle mentioned this pull request Aug 3, 2023

Lock node version to 18.16 vercel/next.js#52894

Merged

ckerr added a commit to electron/electron that referenced this pull request Aug 7, 2023

chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.p…

fd34cad

…atch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0

codebytere mentioned this pull request Jan 2, 2024

chore: drop ada ICU requirement for parsing hostnames electron/electron#40860

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

url: use ada::url_aggregator for parsing urls #47339

url: use ada::url_aggregator for parsing urls #47339

anonrig commented Mar 31, 2023 •

edited

nodejs-github-bot commented Mar 31, 2023

This comment was marked as resolved.

tniessen left a comment

nodejs-github-bot commented Mar 31, 2023

anonrig commented Mar 31, 2023 •

edited

joyeecheung commented Mar 31, 2023 •

edited

anonrig commented Mar 31, 2023 •

edited

bricss commented Mar 31, 2023 •

edited

url: use ada::url_aggregator for parsing urls #47339

url: use ada::url_aggregator for parsing urls #47339

Conversation

anonrig commented Mar 31, 2023 • edited

nodejs-github-bot commented Mar 31, 2023

This comment was marked as resolved.

tniessen left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Mar 31, 2023

anonrig commented Mar 31, 2023 • edited

joyeecheung commented Mar 31, 2023 • edited

anonrig commented Mar 31, 2023 • edited

bricss commented Mar 31, 2023 • edited

anonrig commented Mar 31, 2023 •

edited

anonrig commented Mar 31, 2023 •

edited

joyeecheung commented Mar 31, 2023 •

edited

anonrig commented Mar 31, 2023 •

edited

bricss commented Mar 31, 2023 •

edited