lib: improve esm resolve performance #46652

anonrig · 2023-02-14T16:19:34Z

I couldn't find a benchmark or had the time to write it, but in theory, it should be a lot faster.

cc @nodejs/performance @nodejs/loaders

Refs: nodejs/performance#39

nodejs-github-bot · 2023-02-14T16:19:39Z

Review requested:

@nodejs/loaders
@nodejs/modules

GeoffreyBooth

On quick glance this seems fine, but we need to land #44710 first. Just putting a hold on this until that gets landed.

lib/internal/modules/esm/resolve.js

GeoffreyBooth

I chatted with @anonrig and they offered to rebase #44710 on top of this if this is ready first, which seems fine to me. I’d still like to ensure this doesn’t slow down #44710 landing though; the rebased version of that branch shouldn’t have any new test failures.

Discussion

aduh95

I'd like to see a benchmark for this, but the theory looks solid. This can land before or after the off-thread PR, the conflicts should be minimal :)

arcanis · 2023-02-14T17:52:13Z

Shouldn't it be factorized with the cjs/loader implementation (note that since it isn't in this PR, the esm implementation doesn't has a cache)?

anonrig · 2023-02-14T18:02:16Z

Shouldn't it be factorized with the cjs/loader implementation (note that since it isn't in this PR, the esm implementation doesn't has a cache)?

I agree, but I prefer to do it in a different pull request, and keep this minimal.

mcollina

lgtm

nodejs-github-bot · 2023-02-17T13:34:49Z

CI: https://ci.nodejs.org/job/node-test-pull-request/49633/

nodejs-github-bot · 2023-02-21T22:37:21Z

CI: https://ci.nodejs.org/job/node-test-pull-request/49858/

GeoffreyBooth · 2023-03-13T00:48:01Z

Before we land this shouldn’t there be some attempt at confirming that this does, in fact, make things faster? And especially not slower. Even just running a benchmark on someone’s local machine.

benjamingr

To strengthen @GeoffreyBooth 's point
Making sure benchmarks are run in some form before this lands. Please dismiss my review when that happens @anonrig (or whomever)

anonrig · 2023-04-23T22:58:40Z

@GeoffreyBooth Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1332/

Results

                                                                              confidence improvement accuracy (*)    (**)   (***)
esm/esm-loader-defaultResolve.js specifier='./relative-existing.js' n=1000             *      9.41 %       ±8.24% ±10.97% ±14.29%
esm/esm-loader-defaultResolve.js specifier='./relative-nonexistent.js' n=1000        ***     15.64 %       ±6.89%  ±9.20% ±12.03%
esm/esm-loader-defaultResolve.js specifier='node:os' n=1000                                  -0.28 %       ±8.78% ±11.68% ±15.20%
esm/esm-loader-defaultResolve.js specifier='node:prefixed-nonexistent' n=1000                -3.20 %       ±7.41%  ±9.87% ±12.85%
esm/esm-loader-defaultResolve.js specifier='unprefixed-existing' n=1000               **     13.37 %       ±8.00% ±10.65% ±13.88%
esm/esm-loader-defaultResolve.js specifier='unprefixed-nonexistent' n=1000             *      9.01 %       ±7.15%  ±9.52% ±12.39%

nodejs-github-bot · 2023-04-23T23:52:28Z

Landed in 3e9ed7e

PR-URL: nodejs#46652 Refs: nodejs/performance#39 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Erick Wendel <erick.workspace@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

PR-URL: #46652 Refs: nodejs/performance#39 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Erick Wendel <erick.workspace@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

PR-URL: #46652 Refs: nodejs/performance#39 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Erick Wendel <erick.workspace@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

* chore: upgrade to Node.js v20 * src: allow embedders to override NODE_MODULE_VERSION nodejs/node#49279 * src: fix missing trailing , nodejs/node#46909 * src,tools: initialize cppgc nodejs/node#45704 * tools: allow passing absolute path of config.gypi in js2c nodejs/node#49162 * tools: port js2c.py to C++ nodejs/node#46997 * doc,lib: disambiguate the old term, NativeModule nodejs/node#45673 * chore: fixup Node.js BSSL tests * nodejs/node#49492 * nodejs/node#44498 * deps: upgrade to libuv 1.45.0 nodejs/node#48078 * deps: update V8 to 10.7 nodejs/node#44741 * test: use gcUntil() in test-v8-serialize-leak nodejs/node#49168 * module: make CJS load from ESM loader nodejs/node#47999 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * chore: address changes to CJS/ESM loading * module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652) * bootstrap: optimize modules loaded in the built-in snapshot nodejs/node#45849 * test: mark test-runner-output as flaky nodejs/node#49854 * lib: lazy-load deps in modules/run_main.js nodejs/node#45849 * url: use private properties for brand check nodejs/node#46904 * test: refactor `test-node-output-errors` nodejs/node#48992 * assert: deprecate callTracker nodejs/node#47740 * src: cast v8::Object::GetInternalField() return value to v8::Value nodejs/node#48943 * test: adapt test-v8-stats for V8 update nodejs/node#45230 * tls: ensure TLS Sockets are closed if the underlying wrap closes nodejs/node#49327 * test: deflake test-tls-socket-close nodejs/node#49575 * net: fix crash due to simultaneous close/shutdown on JS Stream Sockets nodejs/node#49400 * net: use asserts in JS Socket Stream to catch races in future nodejs/node#49400 * lib: fix BroadcastChannel initialization location nodejs/node#46864 * src: create BaseObject with node::Realm nodejs/node#44348 * src: implement DataQueue and non-memory resident Blob nodejs/node#45258 * sea: add support for V8 bytecode-only caching nodejs/node#48191 * chore: fixup patch indices * gyp: put filenames in variables nodejs/node#46965 * build: modify js2c.py into GN executable * fix: (WIP) handle string replacement of fs -> original-fs * [v20.x] backport vm-related memory fixes nodejs/node#49874 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * src: avoid copying string in fs_permission nodejs/node#47746 * look upon my works ye mighty and dispair * chore: patch cleanup * [api] Remove AllCan Read/Write https://chromium-review.googlesource.com/c/v8/v8/+/5006387 * fix: missing include for NODE_EXTERN * chore: fixup patch indices * fix: fail properly when js2c fails in Node.js * build: fix js2c root_gen_dir * fix: lib/fs.js -> lib/original-fs.js * build: fix original-fs file xforms * fixup! module: make CJS load from ESM loader * build: get rid of CppHeap for now * build: add patch to prevent extra fs lookup on esm load * build: greatly simplify js2c modifications Moves our original-fs modifications back into a super simple python script action, wires up the output of that action into our call to js2c * chore: update to handle moved internal/modules/helpers file * test: update @types/node test * feat: enable preventing cppgc heap creation * feat: optionally prevent calling V8::EnableWebAssemblyTrapHandler * fix: no cppgc initialization in the renderer * gyp: put filenames in variables nodejs/node#46965 * test: disable single executable tests * fix: nan tests failing on node headers missing file * tls,http2: send fatal alert on ALPN mismatch nodejs/node#44031 * test: disable snapshot tests * nodejs/node#47887 * nodejs/node#49684 * nodejs/node#44193 * build: use deps/v8 for v8/tools Node.js hard depends on these in their builtins * test: fix edge snapshot stack traces nodejs/node#49659 * build: remove js2c //base dep * build: use electron_js2c_toolchain to build node_js2c * fix: don't create SafeSet outside packageResolve Fixes failure in parallel/test-require-delete-array-iterator: === release test-require-delete-array-iterator === Path: parallel/test-require-delete-array-iterator node:internal/per_context/primordials:426 constructor(i) { super(i); } // eslint-disable-line no-useless-constructor ^ TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator)) at new Set (<anonymous>) at new SafeSet (node:internal/per_context/primordials:426:22) * fix: failing crashReporter tests on Linux These were failing because our change from node::InitializeNodeWithArgs to node::InitializeOncePerProcess meant that we now inadvertently called PlatformInit, which reset signal handling. This meant that our intentional crash function ElectronBindings::Crash no longer worked and the renderer process no longer crashed when process.crash() was called. We don't want to use Node.js' default signal handling in the renderer process, so we disable it by passing kNoDefaultSignalHandling to node::InitializeOncePerProcess. * build: only create cppgc heap on non-32 bit platforms * chore: clean up util:CompileAndCall * src: fix compatility with upcoming V8 12.1 APIs nodejs/node#50709 * fix: use thread_local BuiltinLoader * chore: fixup v8 patch indices --------- Co-authored-by: Keeley Hammond <vertedinde@electronjs.org> Co-authored-by: Samuel Attard <marshallofsound@electronjs.org>

PR-URL: nodejs/node#46652 Refs: nodejs/performance#39 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Erick Wendel <erick.workspace@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

anonrig added the performance Issues and PRs related to the performance of Node.js. label Feb 14, 2023

nodejs-github-bot added esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run. labels Feb 14, 2023

GeoffreyBooth previously requested changes Feb 14, 2023

View reviewed changes

GeoffreyBooth added the loaders Issues and PRs related to ES module loaders label Feb 14, 2023

targos reviewed Feb 14, 2023

View reviewed changes

lib/internal/modules/esm/resolve.js Outdated Show resolved Hide resolved

anonrig force-pushed the perf-improve-esm-resolve branch from d5f008b to 76fb5d6 Compare February 14, 2023 16:38

GeoffreyBooth reviewed Feb 14, 2023

View reviewed changes

GeoffreyBooth self-requested a review February 14, 2023 16:45

aduh95 approved these changes Feb 14, 2023

View reviewed changes

mscdex added the needs-citgm PRs that need a CITGM CI run. label Feb 14, 2023

targos approved these changes Feb 15, 2023

View reviewed changes

mcollina approved these changes Feb 15, 2023

View reviewed changes

RafaelGSS approved these changes Feb 15, 2023

View reviewed changes

tony-go approved these changes Feb 16, 2023

View reviewed changes

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 17, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 17, 2023

This was referenced Feb 18, 2023

CI Reliability 2023-02-18 nodejs/reliability#499

Open

CI Reliability 2023-02-19 nodejs/reliability#500

Open

jasnell approved these changes Feb 20, 2023

View reviewed changes

github-actions bot mentioned this pull request Feb 23, 2023

CI Reliability 2023-02-23 nodejs/reliability#504

Open

31 tasks

UlisesGascon approved these changes Mar 12, 2023

View reviewed changes

benjamingr previously requested changes Mar 13, 2023

View reviewed changes

aduh95 approved these changes Apr 23, 2023

View reviewed changes

aduh95 added the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 23, 2023

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 23, 2023

nodejs-github-bot merged commit 3e9ed7e into nodejs:main Apr 23, 2023
27 checks passed

This was referenced Apr 24, 2023

CI Reliability 2023-04-24 nodejs/reliability#545

Open

CI Reliability 2023-04-25 nodejs/reliability#546

Open

CI Reliability 2023-04-26 nodejs/reliability#547

Open

CI Reliability 2023-04-27 nodejs/reliability#548

Open

targos mentioned this pull request May 2, 2023

v20.1.0 release proposal #47820

Merged

danielleadams added the backport-requested-v18.x PRs awaiting manual backport to the v18.x-staging branch. label Jul 4, 2023

codebytere added a commit to electron/electron that referenced this pull request Nov 6, 2023

chore: address changes to CJS/ESM loading

21d21f3

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 14, 2023

chore: address changes to CJS/ESM loading

c1ec1eb

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 15, 2023

chore: address changes to CJS/ESM loading

c7004c1

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 16, 2023

chore: address changes to CJS/ESM loading

b69759d

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 21, 2023

chore: address changes to CJS/ESM loading

efaecec

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 22, 2023

chore: address changes to CJS/ESM loading

42cb35a

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

codebytere added a commit to electron/electron that referenced this pull request Nov 28, 2023

chore: address changes to CJS/ESM loading

10fa7c0

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

targos mentioned this pull request Nov 28, 2023

v18.19.0 release proposal #50953

Merged

codebytere added a commit to electron/electron that referenced this pull request Nov 29, 2023

chore: address changes to CJS/ESM loading

f9e4508

* module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib: improve esm resolve performance #46652

lib: improve esm resolve performance #46652

anonrig commented Feb 14, 2023 •

edited

nodejs-github-bot commented Feb 14, 2023

GeoffreyBooth left a comment

GeoffreyBooth left a comment

aduh95 left a comment

arcanis commented Feb 14, 2023

anonrig commented Feb 14, 2023

mcollina left a comment

nodejs-github-bot commented Feb 17, 2023

nodejs-github-bot commented Feb 21, 2023

GeoffreyBooth commented Mar 13, 2023

benjamingr left a comment •

edited

anonrig commented Apr 23, 2023 •

edited by aduh95

nodejs-github-bot commented Apr 23, 2023

lib: improve esm resolve performance #46652

lib: improve esm resolve performance #46652

Conversation

anonrig commented Feb 14, 2023 • edited

nodejs-github-bot commented Feb 14, 2023

GeoffreyBooth left a comment

Choose a reason for hiding this comment

GeoffreyBooth left a comment

Choose a reason for hiding this comment

aduh95 left a comment

Choose a reason for hiding this comment

arcanis commented Feb 14, 2023

anonrig commented Feb 14, 2023

mcollina left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Feb 17, 2023

nodejs-github-bot commented Feb 21, 2023

GeoffreyBooth commented Mar 13, 2023

benjamingr left a comment • edited

Choose a reason for hiding this comment

anonrig commented Apr 23, 2023 • edited by aduh95

nodejs-github-bot commented Apr 23, 2023

anonrig commented Feb 14, 2023 •

edited

benjamingr left a comment •

edited

anonrig commented Apr 23, 2023 •

edited by aduh95