pnpm 8.9.0 broken for windows dev drivers (ReFS) #7186
Comments
|
@atsjo Do you have an example package.json I can check? I tried it my self and I don't have any error PS D:\code\alotta> pnpm i
Lockfile is up to date, resolution step is skipped
Packages: +1316
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Progress: resolved 1316, reused 1316, downloaded 0, added 1316, done
dependencies:
+ animate.less 2.2.0
+ autoprefixer 10.4.16
+ babel-core 6.26.3
+ babel-eslint 10.1.0
+ babel-loader 8.3.0
+ babel-plugin-lodash 3.3.4
+ babel-plugin-module-resolver 4.1.0
+ babel-plugin-transform-decorators-legacy 1.3.5
+ babel-plugin-transform-runtime 6.23.0
+ babel-polyfill 6.26.0
+ babel-preset-es2015 6.24.1
+ babel-preset-react 6.24.1
+ babel-preset-react-hmre 1.1.1
+ babel-preset-stage-1 6.24.1
+ babel-runtime 6.26.0
+ clean-webpack-plugin 4.0.0
+ core-decorators 0.20.0
+ css-loader 6.8.1
+ css-mqpacker 7.0.0
+ cssnano 5.1.15
+ custom-event-polyfill 1.0.7
+ draft-js 0.11.7
+ ejs 3.1.9
+ eslint 8.51.0
+ eslint-config-airbnb 19.0.4
+ eslint-import-resolver-webpack 0.13.7
+ eslint-plugin-import 2.28.1
+ eslint-plugin-jsx-a11y 6.7.1
+ eslint-plugin-react 7.33.2
+ express 4.18.2
+ express-http-proxy 1.6.3
+ font-awesome 4.7.0
+ fready 1.0.0
+ glob 8.1.0
+ gulp 4.0.2
+ gulp-concat 2.6.1
+ gulp-csslint 1.0.1
+ gulp-cssnano 2.1.3
+ gulp-eol 0.2.0
+ gulp-less 5.0.0
+ gulp-livereload 4.0.2
+ gulp-minify-css 1.2.4
+ gulp-postcss 9.0.1
+ gulp-rename 2.0.0
+ gulp-util 3.0.8
+ happypack 5.0.1
+ highcharts 10.3.3
+ highcharts-solid-gauge 0.1.7
+ history 5.3.0
+ howler 2.2.4
+ imports-loader 4.0.1
+ jquery 3.7.1
+ jquery-ui 1.13.2
+ js-cookie 3.0.5
+ json-loader 0.5.7
+ leftpad 0.0.1
+ less 4.2.0
+ lesshat 4.1.0
+ lodash 4.17.21
+ medium-draft 0.5.18
+ mobx 6.10.2
+ mobx-react 7.6.0
+ moment 2.29.4
+ moment-range 4.0.2
+ moment-timezone 0.5.43
+ password-policy 0.0.3
+ postcss-reporter 7.0.5
+ progress 2.0.3
+ qs 6.11.2
+ raw-loader 4.0.2
+ rc-slider 10.3.0
+ react 18.2.0
+ react-addons-css-transition-group 15.6.2
+ react-addons-shallow-compare 15.6.3
+ react-dnd 16.0.1
+ react-dnd-html5-backend 16.0.1
+ react-dom 18.2.0
+ react-draft-wysiwyg 1.15.0
+ react-dropzone 14.2.3
+ react-grid-layout 1.4.2
+ react-highcharts 16.1.0
+ react-hot-loader 4.13.0
+ react-input-calendar 0.5.4
+ react-lazyload 3.2.0
+ react-measure 2.5.2
+ react-mixin 5.0.0
+ react-responsive 9.0.0-beta.10
+ react-responsive-tabs 4.4.3
+ react-router 6.16.0
+ react-router-dom 6.16.0
+ react-select-plus 1.0.0-rc.3.patch12
+ react-skylight 0.5.1
+ react-sortablejs 6.1.4
+ react-tappable 1.0.4
+ react-tooltip 4.2.21
+ react-virtualized 9.22.5
+ react-waypoint 10.3.0
+ sortablejs 1.15.0
+ style-loader 3.3.3
+ stylelint 14.16.1
+ superagent 8.1.2
+ uglify-js 3.17.4
+ uuid 8.3.2
+ verge 1.10.2
+ webpack-bundle-analyzer 4.9.1
+ webpack-hot-middleware 2.25.4
+ webpack-notifier 1.15.0
+ webpack-split-by-path 2.0.0
+ whatwg-fetch 3.6.19
devDependencies:
+ nan-as 1.6.1
Done in 23.1s |
|
|
Usually don't fail at install, but try pnpm store status, or prune right after install |
|
After a few tests, sometimes when cloning multiple file (or the the same file multiple times) the destination is empty, I'm working on a solution. |
|
8.9.1 fails in the exact same way as 8.9.0... can be triggered with the deps I posted above...
If I didn't delete store first, it worked fine... But then the problem would just surface at some later point while upgrading deps... |
|
If the issue happens after using |
|
didn't use pnpm store prune... but as mentioned, it worked the time I didn't delete store first.... tried several times, it doesn't fail the same way always.... different packages are modified etc, and one time it failed during install... |
why do you say that it fails the exact same way? In the original issue installation was failing. |
|
because it's not deterministic... as mentioned several places above (and in the original issue), it sometimes fails during install, but usually works... but when checking pnpm store status after install, you can se that some packages are changes from the version in store.... |
|
I guess when files that pnpm use during install are changed (package.json), then the install fails... but when some other files change, it's not noticed during install... |
|
So |
|
Made it fail once during install... But it always shows mutated packages after install... |
|
ok, I'll revert it for Windows for now. We can investigate this later. |
|
I just set hard linking in .npmrc for now... |
|
I tracked down the changes in one of the mutated packages by comparing against an install using hardlinks (where it reports no mutated packages)... in a package called cssom@0.3.8 I found a file called CSSStyleSheet.js with the correct size, but content was all zero's when using cloning.... It's not deterministic which files and packages gets the error, but it's fairly easy to detect by running pnpm store status and diffing against a correct install... I can reproduce mutated packages consistently when deleting both store and node_modules, while keeping the lockfile, and then running pnpm i.... |
|
I am releasing 8.9.2, which reverts the change on Windows. |
## Proposed Changes - The main reason for this update is to revert the creation of reflinks on Windows, which doesn't work reliably at the moment (related issue pnpm/pnpm#7186)
|
@atsjo when you were testing the fix, have you update the packageManager field in your |
|
yes... my laptop can easily reproduce this on a dev drive with the procedure described (#7186 (comment) and #7186 (comment)). It was failing with 8.9.0 and 8.9.1, but NOT with 8.9.2, since cloning is now disabled for windows... |
|
I'm running the following commands on my machine, with the pnpm version pnpm i
pnpm store status
pnpm store prune
rimraf node_modules pnpm-lock.yaml && pnpm i
pnpm store statusIn my side almost all the status retrieve
The only status that failed, had this file corrupted, so I'm adding a test for this one in the reflink repo https://pastebin.com/mBqQhYrm I have a function to find files with null bytes, that should help us to find the problem for empty files PS D:\code\modified-store> node .\check-null.mjs node_modules\.pnpm
✖ Null bytes found in the following files:
node_modules\.pnpm\css-tree@2.2.1\node_modules\css-tree\cjs\lexer\match.cjs
node_modules\.pnpm\css-tree@2.3.1\node_modules\css-tree\lib\syntax\node\Selector.js
node_modules\.pnpm\css-tree@2.3.1\node_modules\css-tree\lib\utils\string.js
node_modules\.pnpm\csso@5.0.5\node_modules\css-tree\cjs\lexer\match.cjs
node_modules\.pnpm\node_modules\css-tree\lib\syntax\node\Selector.js
node_modules\.pnpm\node_modules\css-tree\lib\utils\string.js
node_modules\.pnpm\svgo@3.0.2\node_modules\css-tree\lib\syntax\node\Selector.js
node_modules\.pnpm\svgo@3.0.2\node_modules\css-tree\lib\utils\string.js |
|
I cannot use |
Verify latest release
pnpm version
8.9.0
Which area(s) of pnpm are affected? (leave empty if unsure)
Store
Link to the code that reproduces this issue or a replay of the bug
No response
Reproduction steps
running pnpm i: (sometimes fails with below message)
if I get past error above, I can run pnpm store status: (it reports multiple changed packages)
if I run pnpm store prune it also clears all packages in store (like there is no ref)
Describe the Bug
The performance improvements introduced for Windows Dev Drives in 8.9.0 seems to have broken it. It works fine on dev drives with 8.8.0, but I get multiple errors as described in the Repro steps above with 8.9.0. I tried with node 18.18.0 and 20.8.0
Expected Behavior
It works as for 8.8.0 (verified that also pruning works correctly)
Which Node.js version are you using?
18.18.0 and 20.8.0
Which operating systems have you used?
If your OS is a Linux based, which one it is? (Include the version if relevant)
No response
The text was updated successfully, but these errors were encountered: