Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pnpm stuck in windows due to long file path #7355

Closed
1 of 4 tasks
turtleinspace1 opened this issue Nov 29, 2023 · 12 comments · Fixed by #7994
Closed
1 of 4 tasks

Pnpm stuck in windows due to long file path #7355

turtleinspace1 opened this issue Nov 29, 2023 · 12 comments · Fixed by #7994

Comments

@turtleinspace1
Copy link

turtleinspace1 commented Nov 29, 2023

Verify latest release

  • I verified that the issue exists in the latest pnpm release

pnpm version

8.11.0

Which area(s) of pnpm are affected? (leave empty if unsure)

No response

Link to the code that reproduces this issue or a replay of the bug

No response

Reproduction steps

Create a web project in windows
In Package.json look like follow, only install @angular/material
"dependencies": {
"@angular/material": "16.1.3"
},
Pnpm install
Stuck for 6 minutes

Describe the Bug

Pnpm install need a so long time.
After debug the pnpm, the block happen on the follow code( fs.renameSync(oldPath, newPath);)
Pnpm will try to rename the folder for each dependency.
e.g. from
D:\.....s\.pnpm\registry.npmmirror.com+@angular+material@16.1.3_@angular+animations@17.0.4_@angular+cdk@16.1.dbljk25fedifsmmc22b3hvapj4\node_modules\@angular\material_tmp_48644
to
D:\.....\.pnpm\registry.npmmirror.com+@angular+material@16.1.3
@angular+animations@17.0.4_@angular+cdk@16.1.dbljk25fedifsmmc22b3hvapj4\node_modules\@angular\material
The path is so long exceed the limit of Max path in windows.
e.g.
D:....\mui\node_modules.pnpm@angular+material@16.1.3
@angular+animations@17.0.4_@angular+cdk@16.1.3_@angular+common@17.0._p6gpzivryg5zt4x4wcyv73polm\node_modules@angular\material\esm2022\core\selection\pseudo-checkbox
which cause the rename operation stuck

Expected Behavior

For whole project
pnpm install cost 50 minutes, it is too long
npm install will only cost about 6 minutes.

Which Node.js version are you using?

18.17.1

Which operating systems have you used?

  • macOS
  • Windows
  • Linux

If your OS is a Linux based, which one it is? (Include the version if relevant)

No response

@ckolumbus
Copy link

same issue here

@zkochan
Copy link
Member

zkochan commented Dec 9, 2023

I guess we'll optimize it in the next major version of pnpm. I have a PR open already: #7317

However, I don't understand why you get the registry.npmmirror.com+ part in the path. Can you share your lockfile?

@turtleinspace1
Copy link
Author

Thanks, it is very helpful, hope the fix 7317 can be in next major version, pnpm is excellent, but our project can not use the tool due to this problem.
For the registry.npmmirror.com+ part, I also feel weird . but now I can not reproduce it. my current version is 8.12.0
if it happen again, I will provide the lock files.

@knames
Copy link

knames commented Jan 19, 2024

also having this issue, spent 2 days trying to figure out why our project worked fine on my mac and not on my coworkers machines. Turns out our file names are long, and pnpm makes crazy nested path lengths. Even with long file paths turned on, it would still blow up.

@turtleinspace1
Copy link
Author

I find the #7317 is closed and will not be integrated to V9
May I know the reason?
When I use this solution, it can indeed solve my problem.

@zkochan
Copy link
Member

zkochan commented Apr 21, 2024

The issue is still not solved for all users by that fix but it makes the directories in .pnpm less readable. The solution should be to allow long paths on Windows on system level. We should try to fix it with long path prefix: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

@zkochan
Copy link
Member

zkochan commented Apr 21, 2024

@turtleinspace1
Copy link
Author

turtleinspace1 commented Apr 21, 2024

Thanks for advice, I remember that my collogues already try to enable the long path. but it still could not work. But I will try again.
I guess it might be relevant to one of security policy or safety software on our develop environment. Some companies have their own IT department, and they always add some security policy or safety software to develop computer.

When I change the code and set MAX_LENGTH_WITHOUT_HASH to lower value, then the problem is gone.

Maybe it could provide a choice in user side by extra parameter on the pnpm install?
User can decide to use full path or use short path.

@zkochan
Copy link
Member

zkochan commented Apr 22, 2024

To what value do you change MAX_LENGTH_WITHOUT_HASH?

@turtleinspace1
Copy link
Author

To what value do you change MAX_LENGTH_WITHOUT_HASH?
Sorry for later reply.
If MAX_LENGTH_WITHOUT_HASH = 26 it worked
If MAX_LENGTH_WITHOUT_HASH = 60, it also worked

The bigger value, I did not have a try.
Thanks for solving

@zkochan
Copy link
Member

zkochan commented Apr 29, 2024

🚢 9.1.0-0

@turtleinspace1
Copy link
Author

turtleinspace1 commented Apr 30, 2024

I download the 9.1.0-0 and set virtual-store-dir-max-length=80 in .npmrc
Now it works normally, Thanks!
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants