Pnpm autofixing lockfile is not working

[XantreDev]

Verify latest release

• I verified that the issue exists in the latest pnpm release

pnpm version

8.9.2

Which area(s) of pnpm are affected? (leave empty if unsure)

No response

Link to the code that reproduces this issue or a replay of the bug

No response

Reproduction steps

1. Have lockfile with merge confilct
2. Try to install.
3. Fails with error of renaming of random package

Describe the Bug

Produced errors.

 ERR_PNPM_LINKING_FAILED  Error: EPERM: operation not permitted, rename 'D:\Projects\work\Oone.World\trucks-driver-app\packages\location-watcher\node_modules\metro-minify-uglify_tmp_32504' -> 'D:\Projects\work\Oone.World\trucks-driver-app\packages\location-watcher\node_modules\metro-minify-uglify'

or with another:

 ERR_PNPM_LINKING_FAILED  Error: EPERM: operation not permitted, rename 'D:\Projects\work\Oone.World\trucks-driver-app\node_modules\@react-native-firebase\app_tmp_6680' -> 'D:\Projects\work\Oone.World\trucks-driver-app\node_modules\@react-native-firebase\app

npmrc:

shamefully-hoist=true
node-linker=hoisted

Expected Behavior

Install is not failing.
All works correctly for 8.7.6 version

Which Node.js version are you using?

18.18.0

Which operating systems have you used?

• macOS
• Windows
• Linux

If your OS is a Linux based, which one it is? (Include the version if relevant)

No response

[mhamri]

I'm having the same issue, quite giving up on pnpm, every single pnpm install cause a similar error

for me is this

 ERR_PNPM_LINKING_FAILED  Error: EPERM: operation not permitted, rename 'C:\repo\Planally3\frontend-old\node_modules\@nx\eslint-plugin\node_modules\@typescript-eslint\typescript-estree_tmp_9464' -> 'C:\repo\Planally3\frontend-old\node_modules\@nx\eslint-plugin\node_modules\@typescript-eslint\typescript-estree'

always rename is not permitted error. I've added full control on the folder, changed security permission, but nothing helped.

i'm trying to avoid running the pnpm as root
otherwise this issue shouldn't happen

[XantreDev]

Can you copy paste pnpm-lock?

[XantreDev]

Deleting of whole node_modules is not working, too
As workaround you can downgrade pnpm to 8.7.6

[ImUrX]

This happened to me in my windows machine, just after importing a package-lock.json and doing pnpm i, it fixed when I changed to 8.7.6

[zkochan]

Is v8.8.0 the first version where it gets broken?

Here's a related change shipped with 8.8.0: #7117. Maybe this introduced the issue.

[zkochan]

This might fix the issue: zkochan/packages#181. But I couldn't reproduce it.

[ImUrX]

any way i can try it?

[zkochan]

@ImUrX build pnpm from the repository and see if the issue is fixed.

[ImUrX]

I'm no longer able to replicate my issue in the windows machine I had the issues with in 8.10.2 which was where I had issues with 😓.
Not even after deleting the pnpm cache and node_modules in each subpackage.

[mhamri]

I encountered this again today

 ERR_PNPM_LINKING_FAILED  Error: EPERM: operation not permitted, rename 'D:\hustling\Planally\P1\frontend\node_modules\@testing-library\dom\node_modules\aria-query_tmp_20636' -> 'D:\hustling\Planally\P1\frontend\node_modules\@testing-library\dom\node_modules\aria-query'
  pnpm --version
8.10.2

this is the ndjson reporter that could be related

us":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@storybook\\manager-webpack5"}
{"time":1702456165394,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\postcss-modules-extract-imports"}
{"time":1702456165422,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\postcss-modules-local-by-default"}
{"time":1702456165573,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@storybook\\angular"}
{"time":1702456166035,"hostname":"rekab","pid":18444,"level":"error","name":"pnpm","code":"ERR_PNPM_LINKING_FAILED","name":"pnpm","err":{"name":"pnpm","message":"Error: ENOENT: no such file or directory, scandir 'D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@types\\jest_tmp_18444\\node_modules'","code":"ERR_PNPM_LINKING_FAILED","stack":"pnpm: Error: ENOENT: no such file or directory, scandir 'D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@types\\jest_tmp_18444\\node_modules'\n    at Worker.<anonymous> (C:\\snapshot\\pnpm\\pnpm\\dist\\pnpm.cjs)\n    at Object.onceWrapper (node:events:652:26)\n    at Worker.emit (node:events:537:28)\n    at MessagePort.<anonymous> (node:internal/worker:233:53)\n    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:645:20)\n    at exports.emitMessage (node:internal/per_context/messageport:23:28)"}}
{"time":1702456166158,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\postcss-modules-scope"}
{"time":1702456166188,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\postcss-modules-values"}
{"time":1702456166286,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@testing-library\\cypress"}
{"time":1702456166440,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@angular\\compiler-cli"}
{"time":1702456166465,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@types\\pluralize"}
{"time":1702456166582,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@types\\node"}
{"time":1702456167460,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"copy","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@angular\\cli"}
{"time":1702456167471,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\mathjs"}
{"time":1702456167520,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\@typescript-eslint\\parser"}
{"time":1702456167553,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\cypress-file-upload"}
{"time":1702456167584,"hostname":"rekab","pid":18444,"level":"debug","name":"pnpm:progress","method":"hardlink","requester":"D:\\hustling\\Planally\\P1\\frontend","status":"imported","to":"D:\\hustling\\Planally\\P1\\frontend\\node_modules\\cypress-network-idle"}

[sharky98]

This might fix the issue: zkochan/packages#181. But I couldn't reproduce it.

No idea if this is a permanent fix, but after manually patching the dist/pnpm.cjs file with this code pnpm install finally worked after trying for about 3 hours to find what was the culprit, I still suspect the antivirus, but I have no control over it due to corporate policies.

For the record, on Windows 10

EDIT: Well, It may happens it was just a lucky run. I tried adding a package and I keep getting again the ERR_PNPM_LINKING_FAILED  Error: EPERM: operation not permitted, rename again on a new package. It did work though on the 3rd run I made...

[zkochan]

I assume this started happening when we switched to sync filesystem operations. In the past pnpm used async fs operations but now we use sync operations in multiple worker threads that run in parallel. The async version of the rename function is doing the retry after a small delay, which might result in better success rate. The sync version retries immediately. We can try to add "busy waiting" to the sync version.

[sharky98]

I assume this started happening when we switched to sync filesystem operations. In the past pnpm used async fs operations but now we use sync operations in multiple worker threads that run in parallel. The async version of the rename function is doing the retry after a small delay, which might result in better success rate. The sync version retries immediately. We can try to add "busy waiting" to the sync version.

If this change happened this year, this could be it. Earlier this year I was not having issues at all (don't remember the version though). Then it started popping up very often, never on the same file.

With the proposed change above to retry, it continues to do the error, but way less frequently!

To me it seems related to the number of packages installed at once. Chances are that the process reading the tmp files is not able to keep up the pace with pnpm renaming them (or the opposite, pnpm is too slow to rename them before the process kicks in to read them 🤷)

When adding very few packages, it seems less frequent, but when doing a fresh pnpm install on a big list of packages, it does happen a lot more.