Error: spawn E2BIG when running postinstall scripts

[douglasward]

When upgrading from version 7.18.2 to 7.19.0, pnpm install errors out when running the postinstall scripts with the following error:

node:internal/child_process:413
    throw errnoException(err, 'spawn');
    ^
Error: spawn E2BIG
    at ChildProcess.spawn (node:internal/child_process:413:11)
    at spawn (node:child_process:713:9)
    at spawn (/usr/local/lib/node_modules/pnpm/dist/pnpm.cjs:104514:19)
    at runCmd_ (/usr/local/lib/node_modules/pnpm/dist/pnpm.cjs:117817:20)
    at /usr/local/lib/node_modules/pnpm/dist/pnpm.cjs:117761:11
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {
  errno: -7,
  code: 'E2BIG',
  syscall: 'spawn'
}

It doesn't seem to matter which postinstall script is run, any from any dependency or our own one will suffice.

Our .npmrc file is as follows

node-linker=hoisted
shared-workspace-lockfile = false

We need node-linker=hoisted as we have some angular projects in our mono-repo. Unsetting this seems to fix the problem but breaks the application builds.

These to pull requests seem that they might be related (included in the 7.19 release):

• feat: store locations of deps when node-linker is set to hoisted #5795
• perf: do not build the same dependency multiple times when node-linker is hoisted #5814

pnpm version: 7.19.0 and upwards

Additional information:

• node -v prints: 16.17.1
• Windows, macOS, or Linux?: Tested on MacOS and on linux (Linux e545ebd99b57 4.19.0-17-amd64 #1 SMP Debian 4.19.194-1 (2021-06-10) x86_64 x86_64 x86_64 GNU/Linux)

[await-ovo]

Could you provide a minimal repo to reproduce this issue, I just tried it with the following project:

// packages/foo/package.json
{
  "name": "@pkg/foo",
  "dependencies": {
    "got": "^12.5.3",
    "ava": "^5.2.0",
    "prisma": "latest",
    "ts-node": "latest"
  }
}
// packages/bar/package.json
{
  "name": "@pkg/bar",
  "dependencies": {
    "prisma": "latest"
  }
}

// .npmrc
node-linker=hoisted
shared-workspace-lockfile=false

seems postinstall works find on pnpm v7.27.0:

[douglasward]

yes, I've managed to boil it down to a lock file with the following packages:

{
  "name": "test",
  "devDependencies": {
    "@angular-devkit/build-angular": "^13.3.0",
    "@angular-devkit/core": "^13.3.0",
    "@angular/cli": "^13.3.0",
    "@angular/compiler-cli": "^13.3.0",
    "@nguniversal/builders": "^13.1.0",
    "jest": "^27.4.7",
    "jest-preset-angular": "^11.1.1"
  }
}

I also noticed that I can only reproduce it on linux, not on macOS.

I have attached a minimal repo to reproduce. If you use the Dockerfile to build an image it will throw the error, e.g. with docker build -t pnpm .

pnpm.zip

[douglasward]

My colleague can reproduce it on his linux machine just with

"@angular-devkit/build-angular": "^13.3.0",
"@angular-devkit/core": "^13.3.0",

and no lockfile

[VeselyT]

This seems to be caused by PATH environment variable (generated somewhere in npm-lifecycle package I think) that contains paths to all .bin folders in all nested node_modules?

It seems to be caused by this parameter that contains all those paths

pnpm/exec/lifecycle/src/runLifecycleHook.ts

Line 66 in f512150

extraBinPaths: opts.extraBinPaths ?? [],

(in our case about 3500 entries). Those paths look like this

Hope this will help to fix this issue.

[particle4dev]

@zkochan any update on this issue?

[tomikpik]

+1 for this, it affects our production workload :(((( Could you please take a look at it?

[LuckaFEL]

Adding my vote for this. It's causing some serious disruptions in our production workflow. Mind giving it a look?