New --import API breaks worker threads

[zner0L]

Problem

I was tracking #330 closely and was very happy for you to fix it with #337 and release so quickly, but I just now tested it with loading worker threads and that fails with ERR_UNKNOWN_FILE_EXTENSION.

Minimal example:

import { isMainThread, Worker } from 'worker_threads';

if (isMainThread) {
    const worker = new Worker(new URL(import.meta.url));
} else console.log('Hello from worker thread!');

Result:

yarn tsx test.ts
yarn run v1.22.19
$ /<redacted>/node_modules/.bin/tsx test.ts

node:internal/event_target:1083
  process.nextTick(() => { throw err; });
                           ^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /<redacted>/test.ts
    at __node_internal_captureLargerStackTrace (node:internal/errors:497:5)
    at new NodeError (node:internal/errors:406:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:99:9)
    at defaultGetFormat (node:internal/modules/esm/get_format:142:36)
    at defaultLoad (node:internal/modules/esm/load:120:20)
    at ModuleLoader.load (node:internal/modules/esm/loader:396:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:278:56)
    at new ModuleJob (node:internal/modules/esm/module_job:65:26)
    at #createModuleJob (node:internal/modules/esm/loader:290:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:248:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:229:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:315:23)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:66:12)
Emitted 'error' event on Worker instance at:
    at [kOnErrorMessage] (node:internal/worker:326:10)
    at [kOnMessage] (node:internal/worker:337:37)
    at MessagePort.<anonymous> (node:internal/worker:232:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:807:20)
    at exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

Node.js v20.8.0

Expected behavior

If I run the minimal example with tsx v3.13.0 the output is as expected (though with warnings):

❯ yarn tsx test.ts
yarn run v1.22.19
$ /<redacted>/node_modules/.bin/tsx test.ts
(node:35991) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("file%3A///<redacted>/node_modules/tsx/dist/loader.mjs", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)
(node:35991) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("file%3A///<redacted>/node_modules/tsx/dist/loader.mjs", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)
Hello from worker thread!
Done in 0.25s.

Minimal reproduction URL

Included in issue

Version

v3.14.0

Node.js version

v20.8.0

Package manager

yarn

Operating system

Linux

Contributions

• I plan to open a pull request for this issue
• I plan to make a financial contribution to this project

[waynebloss]

I tried passing options to the Worker constructor such as env: { NODE_OPTIONS: "--import tsx" } and execArgv: ["--import", "tsx"] but neither of those worked and then I found nodejs/node#47747 (ESM loaders cannot be defined via Worker option...)

I think this is really a Node bug or design issue. If a loader was specified at the top level, Node should be using that automatically for Worker threads.

[waynebloss]

Other notes

• Node has a plan to support identifying external modules, e.g. typescript see module: improve external format support nodejs/node#49704
• There is a discussion about missing use cases for the module.register() API at Discussion: Missing use cases with module customization hooks nodejs/node#50392
• I found the above 2 notes from a comment in a proposal to use the ESM loader everywhere, see Proposal: Have the ESM loader handle all entry points nodejs/node#50356 (comment)

[alanhoff]

I've found a temporary workaround: while developing, you'll want your worker to load tsx/cli instead, then pass the actual script you wanna run as an argv, like so:

	const tsx = new URL(import.meta.resolve('tsx/cli'));
	const worker = new Worker(tsx, {env: SHARE_ENV, argv: ['./your/script.ts']});

[privatenumber]

Looked into this. Sounds like this is a limitation on Node's end? Marking this as blocked

[SynthLuvr]

This is an issue for node v18.19.0 too

[privatenumber]

🛑 Want this fixed?

This is currently a limitation in Node.js. The issue was closed without resolution: nodejs/node#47747

They currently don't think the performance cost is worth it, but are open to learning about more use-cases.

I have requested for it to be re-opened. If you'd like loaders to be supported in Workers,

your best course of action is to share your use-case there.

Again, don't just say "plz fix". Share your use-case in nodejs/node#47747.

[privatenumber]

This is blocked by Node.js. Please contribute constructively in nodejs/node#47747

Looks like it's been re-opened with a potential solution that could use help.

Locking thread until resolved in Node.