Performance and sequencing characteristics of loader thread

[cspotcode]

Related to nodejs/node#44710 but I want to extract here so it can outlive that PR

It would be great to have written down the expected sequencing behavior of the loader thread. I'm not sure if "sequencing" is the correct term, but I would like to understand how the thread's workload tasks run concurrently or sequentially.

Past revisions of the off-thread PR forced certain tasks to run sequentially, where you would otherwise assume they are happening concurrently. Hopefully that is no longer the case. I'm hoping to verify by asking this question.

Scenarios

sync request from one thread blocking sync request from another

• Given a main thread, a user thread, and the ESM loader thread
• the main thread makes a blocking import.meta.resolve calls to the loader thread which must communicate with some-remote-process://takes-100ms-to-load/package.json
• the user thread makes a blocking import.meta.resolve calls to the loader thread which must communicate with some-remote-process://also-takes-100ms-to-load/package.json
• Do those tasks execute concurrently within the esm loader thread? Or sequentially?

When (future) main thread makes 20x synchronous require.resolve calls, can each run synchronously in the loader thread? Or does the loader thread need to spin its event loop, handling a single request every tick?

• today, require.resolve behavior can be tweaked in-thread, synchronously (fast)
• will future require.resolve tweaks require spinning loader thread's event loop (slow) or can that also be done synchronously?

When loader thread is blocked by atomics because it has no work, can a message posted to globalPreload port wake it up?

[GeoffreyBooth]

I don’t think anything on that PR affects require.resolve; perhaps you meant to say import.meta.resolve there?

As far as I understand things, import.meta.resolve on the off-thread branch functions kind of like require.resolve or fs.readSync etc., where it synchronously blocks the main thread while it runs. So that sounds like it would be sequential.

I’m not sure what you mean by “the user thread;” normal Node apps have only one thread, and if --loader is used they’ll have two, but that’s it unless user code creates additional Worker threads.

For the rest of it, I think the best way to get to the bottom of these questions would be for you to add some tests to the off-thread branch. Cover the various scenarios you’re discussing here, and have the expected result of the test match what you think should happen, and if that isn’t what occurs then we can discuss.

[aduh95]

• Given a main thread, a user thread, and the ESM loader thread

I think the scenario cannot happen with the current PR, as the user thread would have its own ESM loader thread. It's still interesting to discuss the outcomes in case this is ever implemented (but I suppose would need to be opt-in).

• the main thread makes a blocking import.meta.resolve calls to the loader thread which must communicate with some-remote-process://takes-100ms-to-load/package.json
• the user thread makes a blocking import.meta.resolve calls to the loader thread which must communicate with some-remote-process://also-takes-100ms-to-load/package.json

Currently the worker assumes there can be at most one sync request at a time, so we would need to find some clever way to work around that.

When (future) main thread makes 20x synchronous require.resolve calls, can each run synchronously in the loader thread? Or does the loader thread need to spin its event loop, handling a single request every tick?

Currently we are using "message" events, so it would have the same limitation as with other events (I'm not an expert here, but I think if there are 20+ messages in the queue, there would all be handled in the same tick)

When loader thread is blocked by atomics because it has no work, can a message posted to globalPreload port wake it up?

That's an interesting use case, could you write a test for that?

[cspotcode]

I don’t think anything on that PR affects require.resolve; perhaps you meant to say import.meta.resolve there?

I'm thinking ahead to the future where CJS can be handled by loaders, too. This is confusing, I'll try to update the OP to make this clearer.

I think the scenario cannot happen with the current PR, as the user thread would have its own ESM loader thread.

I see. Is a future change going to collapse the multiple loader threads into a single loader thread, shared across all main and user threads? What's the current plan?

[aduh95]

Is a future change going to collapse the multiple loader threads into a single loader thread, shared across all main and user threads? What's the current plan?

Currently, you can specify whichever loaders for you worker thread, it doesn't inherit from the parent thread. This feature is not going away, as it seems a common enough use case to have a different loading mechanism for a Worker. That being said, we could have an opt-in option to share loaders, this only needs a champion I think.

[cspotcode]

Ok, I don't necessarily want it to be one way or the other, so long as the docs mention its behavior, whatever that is.

[cspotcode]

When loader thread is blocked by atomics because it has no work, can a message posted to globalPreload port wake it up?

That's an interesting use case, could you write a test for that?

The use-case is to avoid creating 2x copies of a transpiler and static analysis service.

This service is used to transpile both require()d CJS and import-ed ESM, and also provide static analysis diagnostics for all those files, where diagnostics in one file often require analyzing the source of other files. This is how typechecking works.

require() can be hooked by require.extensions. If we are also running an ESM loader, then we have a typechecker service running in the loader thread. To avoid creating a duplicate typechecker in the main thread, we delegate all typechecking and transpilation to the ESM loader thread. require.extensions hooks make blocking calls to the loader thread, requesting both transpiled source and diagnostic messages. This requires atomics and globalPreload port messages. These messages must wake up the loader thread.

I'm also curious if quibble needs this: #124 (comment)

[aduh95]

When loader thread is blocked by atomics because it has no work, can a message posted to globalPreload port wake it up?

FYI the version that has landed is a bit different from the time we discussed this, now the worker never locks itself. Having a test would still be interesting I think, if someone wants to contribute that, that would be awesome.