Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jest-worker 27.4.0 breaks webpack builds with node options #14859

Closed
MirKml opened this issue Nov 29, 2021 · 12 comments
Closed

jest-worker 27.4.0 breaks webpack builds with node options #14859

MirKml opened this issue Nov 29, 2021 · 12 comments

Comments

@MirKml
Copy link

MirKml commented Nov 29, 2021 •
edited

Bug report

When latest webpack is used with latest jest - 27.4.0, it breaks production builds with custom node options. e.g.
node --max_old_space_size=2768 node_modules/webpack-cli/bin/cli.js --mode production

Result is

ERROR in 497.a3d1.js from Terser
Error [ERR_WORKER_INVALID_EXEC_ARGV]: Initiated Worker with invalid execArgv flags: --max_old_space_size=2768
    at new NodeError (internal/errors.js:322:7)
    at new Worker (internal/worker.js:196:13)
    at ExperimentalWorker.initialize (/permissions/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:149:20)
    at new ExperimentalWorker (/permissions/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:145:10)
    at WorkerPool.createWorker (/permissions/node_modules/jest-worker/build/WorkerPool.js:44:12)
    at new BaseWorkerPool (/permissions/node_modules/jest-worker/build/base/BaseWorkerPool.js:127:27)
    at new WorkerPool (/permissions/node_modules/jest-worker/build/WorkerPool.js:30:1)
    at new Worker (/permissions/node_modules/jest-worker/build/index.js:167:26)
    at getWorker (/permissions/node_modules/terser-webpack-plugin/dist/index.js:288:9)
    at /permissions/node_modules/terser-webpack-plugin/dist/index.js:389:41

Problem is with latest jest-worker.
There is installed tree with npm

$ npm ls jest-worker
permissions@1.0.0 ./src
├─┬ babel-jest@27.4.0
│ └─┬ @jest/transform@27.4.0
│   └─┬ jest-haste-map@27.4.0
│     └── jest-worker@27.4.0 deduped
├─┬ jest@27.4.0
│ └─┬ @jest/core@27.4.0
│   ├─┬ @jest/reporters@27.4.0
│   │ └── jest-worker@27.4.0 deduped
│   └─┬ jest-runner@27.4.0
│     └── jest-worker@27.4.0 deduped
└─┬ webpack@5.64.4
  └─┬ terser-webpack-plugin@5.1.4
    └── jest-worker@27.4.0

If the current behavior is a bug, please provide the steps to reproduce.

Install latest webpack with latest jest
Use some simple webpack config which utilizes production build. I can try it later for minimal setup if necessary.

What is the expected behavior?

Production builds work as expected. Lowering jest-worker dependency can be solution.

Other relevant information:
webpack version: 5.64.4
Node.js version: 16.13.0
Operating System: Windows, Linux
Additional tools: jest@27.4.0
@MirKml
Copy link
Author

MirKml commented Nov 29, 2021

Temporary workaround - install jest-worker@27.3.* by hand. Then webpack production builds with node options works as usually.
$ npm i -D jest-worker@27.3.1
$ node --max_old_space_size=2768 node_modules/webpack-cli/bin/cli.js --mode production

assets by info 5.72 MiB [immutable]
  assets by path *.js 3.07 MiB 4 assets
  assets by path *.svg 1.64 MiB 4 assets
  assets by path *.eot 343 KiB 3 assets
  assets by path *.ttf 342 KiB 3 assets
  assets by path *.woff 196 KiB 3 assets
....
Entrypoints:
  main (3.14 MiB)
      main.css
      main.1ed2.js

webpack 5.64.4 compiled with 2 warnings in 25863 ms

But I think it's up to webpack developers fix erroneous dependency. Or create issue on jest.
It's news to me, that jest-worker is wepback dependency.

@MirKml MirKml changed the title jest-worker in version 27.4.0 breaks webpack builds with node options jest-worker 27.4.0 breaks webpack builds with node options Nov 29, 2021
@timoschlueter
Copy link

I can confirm this broke our pipelines today. We fixed it with the workaround from @MirKml.
Not perfect, but works for now. Thank you!

@alexander-akait
Copy link
Member

Fixed https://github.com/facebook/jest/releases/tag/v27.4.1, please update jest

@anagh416
Copy link

anagh416 commented Dec 2, 2021

after this fix has been moved , I m getting this error (from 29th Nov)
the jest-worker I have is 27.4.2 seems like related to this fix
21:57:38 ERROR in 319.0a46e60283f8bec09734.manager.bundle.js from Terser
21:57:38 Error [ERR_WORKER_INVALID_EXEC_ARGV]: Initiated Worker with invalid NODE_OPTIONS env variable: --max-http-header-size= is not allowed in NODE_OPTIONS
21:57:38 at new Worker (internal/worker.js:164:13)
21:57:38 at ExperimentalWorker.initialize (/bld/workspace/_Pipeline2_NX_Upgrade_forCypress@2/flex/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:149:20)
21:57:38 at new ExperimentalWorker (/bld/workspace/_Pipeline2_NX_Upgrade_forCypress@2/flex/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:145:10)
21:57:38 at WorkerPool.createWorker (/bld/workspace/_Pipeline2_NX_Upgrade_forCypress@2/flex/node_modules/jest-worker/build/WorkerPool.js:44:12)
21:57:38 at new BaseWorkerPool (/bld/workspace/_Pipeline2_NX_Upgrade_forCypress@2/flex/node_modules/jest-worker/build/base/BaseWorkerPool.js:127:27)
2

@kelly-tock
Copy link

I tried latest jest, did not fix the issue.

@kelly-tock
Copy link

suggested workaround did not work either. getting this:

Initiated Worker with invalid execArgv flags: --max-old-space-size=4096
Error [ERR_WORKER_INVALID_EXEC_ARGV]: Initiated Worker with invalid execArgv flags: --max-old-space-size=4096
    at new NodeError (internal/errors.js:322:7)
    at new Worker (internal/worker.js:196:13)
    at ExperimentalWorker.initialize (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:149:20)
    at new ExperimentalWorker (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/workers/NodeThreadsWorker.js:145:10)
    at WorkerPool.createWorker (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/WorkerPool.js:44:12)
    at new BaseWorkerPool (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/base/BaseWorkerPool.js:127:27)
    at new WorkerPool (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/WorkerPool.js:30:1)
    at new Worker (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/node_modules/jest-worker/build/index.js:167:26)
    at getWorker (/Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/dist/index.js:391:9)
    at /Users/kelly/tock/admin/consumer/node_modules/terser-webpack-plugin/dist/index.js:494:41

@kelly-tock
Copy link

running the command via yarn script:

"webpack_client_prod": "NODE_ENV=production npx -n '--max-old-space-size=4096' -n '--trace-deprecation' webpack --color --config ./webpack/webpack.config.prod.client.js",

@MirKml
Copy link
Author

MirKml commented Dec 3, 2021

In mentioned jest PR which fixed issue, there is discussion about underscores, dashes. Maybe it helps you.

My mentioned node option --max_old_space_size is fixed.

@alexander-akait
Copy link
Member

Even time when you write - Not fixed, please use npm ls jest-worker, and double check you have update version

@DzmVasileusky
Copy link

DzmVasileusky commented Dec 9, 2021 •
edited

@alexander-akait @JonWalsten

I've checked the code of worker after the fix and it seems that you are filtering v8 node options only for max_old_space_size (max-old-space-size).

But it seems other options are causing the same issue.
According to Node docs:

execArgv <string[]> List of node CLI options passed to the worker. V8 options (such as --max-old-space-size) and options that affect the process (such as --title) are not supported. If set, this is provided as process.execArgv inside the worker. By default, options are inherited from the parent thread

https://nodejs.org/api/worker_threads.html#new-workerfilename-options

And there is a very long list of such options.
Build is failing for max-http-header-size=20000 in our case (and I see the same issue in comments above)

@alexander-akait
Copy link
Member

alexander-akait commented Dec 9, 2021 •
edited

Please open an issue in jest repo

@DzmVasileusky
Copy link

DzmVasileusky commented Dec 9, 2021 •
edited

@anagh416 @kelly-tock

They are already working on it
jestjs/jest#12128

donovanglover added a commit to donovanglover/make-synchronized that referenced this issue Apr 15, 2024
@donovanglover
fix(worker): Don't parse invalid NODE_OPTIONS
a1ba2e1 
This was causing downstream projects like prettier-synchronized to break
when encountering V8 options like `--max-old-space-size`.

The option itself shouldn't be needed since execArgv values are
inherited from the parent thread by default.

See: https://nodejs.org/api/worker_threads.html#new-workerfilename-options

Related: webpack/webpack#14859
@donovanglover donovanglover mentioned this issue Apr 15, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@timoschlueter @alexander-akait @DzmVasileusky @MirKml @anagh416 @kelly-tock