fix: avoid optimized deps during build in worker bundles

[patak-dev]

Description

After #11218, we introduced a regression when deps optimization is enabled during build. Since v3, the way deps optimization works is that dependencies imported in workers are optimized together with the dependencies of the main app. For this to work, given that each worker is bundled by a different rollup process, we were awaiting for all the workers to be bundled before doing the optimization run with esbuild (as we need to know during build every dep that should be optimized before, we don't have reloads at build time). We had then a registerWorkerSource scheme so we avoid waiting for the worker id, and instead wait for the registered ids inside the worker. For this scheme to work, all the workers need to be processed in parallel and #11218 serialized the workers. This means that there could potentially be a deadlock if there are deps imported by several workers.

Unrelated to #11218, we have an issue with maxParallelFileOps. Rollup is willing to give us better tools to avoid this problem (see rollup/rollup#4985). Using delayLoading would be very clean, but if we need to also use it to await for the worker bundles, we are back to registering worker sources and adding extra complexity. It should be doable though, but we also need to revert #11218.

This PR explores not using deps optimization when bundling workers. I still don't know if this is acceptable, but I don't know how we can work out optimizing deps for them if they aren't bundled in parallel. Even if we force the user to include all the deps in the worker by hand, it won't work. Sending the PR in case others would like to also check this problem, I don't think we should merge it yet.

---

What is the purpose of this pull request?

• Bug fix
• New Feature
• Documentation update
• Other

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[patak-dev]

Closing this as we have merged #14685, and we are back at doing parallel builds for workers. Optimizing dependencies in workers is possible now, but I no longer think that this is what we'll be recommending in the future. With rolldown, there shouldn't be the need for pre-bundling during build, or it will be tackled directly by its own caching.