vm: support using the default loader to handle dynamic import()

[joyeecheung]

This patch adds support for using vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER as the
importModuleDynamically option in all vm APIs that take this option except vm.SourceTextModule. This allows users to have a shortcut to support dynamic import() in the compiled code without missing the compilation cache if they don't need customization of the loading process. We emit an experimental warning when the import() is actually handled by the default loader through this option instead of requiring --experimental-vm-modules.

const { Script, constants } = require('node:vm');
const { resolve } = require('node:path');
const { writeFileSync } = require('node:fs');

// Write test.js and test.txt to the directory where the current script
// being run is located.
writeFileSync(resolve(__dirname, 'test.mjs'),
              'export const filename = "./test.json";');
writeFileSync(resolve(__dirname, 'test.json'),
              '{"hello": "world"}');

// Compile a script that loads test.mjs and then test.json
// as if the script is placed in the same directory.
const script = new Script(
  `(async function() {
    const { filename } = await import('./test.mjs');
    return import(filename, { with: { type: 'json' } })
  })();`,
  {
    filename: resolve(__dirname, 'test-with-default.js'),
    importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER,
  });

// { default: { hello: 'world' } }
script.runInThisContext().then(console.log);

In addition this refactors the documentation for
importModuleDynamically and adds a dedicated section for it with examples.

vm.SourceTextModule is not supported in this patch because it needs additional refactoring to handle initializeImportMeta, which can be done in a follow-up.

Fixes: #51154

[nodejs-github-bot]

Review requested:

• @nodejs/loaders
• @nodejs/vm
• @nodejs/web-infra

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/56421/

[GeoffreyBooth]

A few small things, but mostly looks great! Thank you for doing this. The last time I tried to use --experimental-vm-modules I was overwhelmed by the amount of boilerplate required; I’m glad to see a lot of that going away.

[GeoffreyBooth]

This seems like it’s a CommonJS script? But it’s a bit of an awkward example since in CommonJS most users would use require to get a JSON file, rather than dynamic import. Maybe change it to a dynamic import of an ES module, with a comment above like // Import ES module into CommonJS code?

I assume we can’t show how this would be used with an ESM script because this doesn’t yet support SourceTextModule?

[joyeecheung]

This is not a CommonJS script, vm.Script is just a script. If you run it in a new context, there is no require() at all (there also isn't going to be process, or URL, or anything not implemented by V8).

[GeoffreyBooth]

URL string, or URL instance? Generally we don’t allow a path to be either a path string or an URL string, because of ambiguity. #48994

[joyeecheung]

Only strings are allowed as resource names, passing URL instances result in an error, this is the compilation API where the name is going to be used for stack traces and must be primitives.

[joyeecheung]

Also note that this is not necessarily a path, it can be a http:// URL. As long as import('file:///path/to/foo') is allowed by the default loader, this is allowed. Whenever that becomes disallowed by the default loader, the default loader can throw, and this documentation can be updated.

[GeoffreyBooth]

We can’t do this. I proposed this in #48994 but it was shot down because you can, in fact, have a path that begins with file://; and it’s actually not all that uncommon, when people mirror URLs in a local filesystem. The logic needs to be “if it’s an instance of the URL class, parse as URL; else parse strings as paths.” cc @aduh95

[joyeecheung]

We do not accept URL in the API, only the strings. This is the current behavior and disallowing file:// paths is semver major (because the filename is part of the stable API).

Also note that only strings or other primitives can end up here after the hands of V8. The default loader can also choose to throw for a file:/// string in the future, and we can update the docs when that happens. For now this is just mimic-ing the existing behavior.

[joyeecheung]

Updated to simply return the string as-is instead of parsing it at all when it looks like a URL. However the default loader treat a module with a file:// URL string name is up to the default loader, we are only passing the name over.

[GeoffreyBooth]

This function could really use a JSDoc, if you know what it should be. What is filename? Is it a path string? Because normalizeReferrerURL returns an URL string; shouldn’t it be one or the other?

This feature should support data URLs and --experimental-network-imports, so path strings should probably be normalized to URL strings so that non-file:// URLs work.

[joyeecheung]

This function could really use a JSDoc, if you know what it should be.

I don't know fully what this does but filename is supposed to be always a string because it's used as keys to SafeMaps below.

shouldn’t it be one or the other?

The current behavior is that "it doesn't matter as long as this string identifies a module that contains the sourceURL that V8 gives back to us". I think there can be some cache misses caused by this but that's probably out of scope of this PR, because this PR just tries to preserve the existing mapping behavior in normalizeReferrerURL.

This feature should support data URLs and --experimental-network-imports, so path strings should probably be normalized to URL strings so that non-file:// URLs work.

Supporting those for source maps seems to be out of scope for this PR? Again this is just preserving the existing behavior. Whether additional normalization should happen and how that should work with other components should probably be discussed elsewhere.

[legendecas]

The filename here can either be an absolute path string (for CJS) or a URL string (for ESM) and is normalized to a URL string by normalizeReferrerURL. I can add JSDoc for source map related functions in a follow-up PR.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/56446/

[joyeecheung]

Ping @GeoffreyBooth did I answer your question about URL strings? The vm APIs only allows strings as identifiers and after the hands of V8 the resource names must be strings. I don't think URL is that big of an issue for this change. Whatever the vm API supports, it will continue to support. Disallowing file:// strings for resource names in vm APIs would be a semver-major that should be discussed elsewhere.

[nodejs-github-bot]

Landed in ad0bcb9