Lockfile generated after pnpm patch cannot be used with --no-frozen-lockfile on CI for Windows

[natemoo-re]

pnpm version: 7.4.1

Code to reproduce the issue:

I opened a PR to replace patch-package with pnpm patch and pnpm patch-commit, but our CI is failing on Windows only.

Our lockfile was generated on macOS, so I'm wondering if the paths saved to the lockfile https://github.com/withastro/astro/pull/3747/files#diff-32824c984905bb02bc7ffcef96a77addd1f1602cff71a11fbbfdd7f53ee026bbR3-R10 are the culprit here?

Expected behavior:

CI environments defaulting to --no-frozen-lockfile should still work on Windows.

Actual behavior:

On Windows only, we're getting this error when running pnpm install in CI.

ERR_PNPM_FROZEN_LOCKFILE_WITH_OUTDATED_LOCKFILE  Cannot perform a frozen installation because the lockfile needs updates

Note that in CI environments this setting is true by default. If you still need to run install in such cases, use "pnpm install --no-frozen-lockfile"

Additional information:

• node -v prints: 14, 16
• Windows, macOS, or Linux?: Windows only

[BlackHole1]

If command is: pnpm install --no-frozen-lockfile in Windows. pnpm-lock.yaml file diff is:

diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index a6b1d0a9..434d9ae8 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -2,10 +2,10 @@ lockfileVersion: 5.4

 patchedDependencies:
   '@changesets/cli@2.23.0':
-    hash: kcozqtpxuwjzskw6zg5royevn4
+    hash: kuwjnalnvawb7qsar2xshwwayi
     path: patches/@changesets__cli@2.23.0.patch
   '@changesets/assemble-release-plan@5.1.3':
-    hash: t3ywtunilq53oqooixy7kemkoi
+    hash: 6bkrbdwcq4wpuel6jqtb3kupwi
     path: patches/@changesets__assemble-release-plan@5.1.3.patch
 
 importers:
@@ -35,9 +35,9 @@ importers:
     dependencies:
       '@astrojs/webapi': link:packages/webapi
     devDependencies:
-      '@changesets/assemble-release-plan': 5.1.3_t3ywtunilq53oqooixy7kemkoi
+      '@changesets/assemble-release-plan': 5.1.3_6bkrbdwcq4wpuel6jqtb3kupwi
       '@changesets/changelog-github': 0.4.4
-      '@changesets/cli': 2.23.0_kcozqtpxuwjzskw6zg5royevn4
+      '@changesets/cli': 2.23.0_kuwjnalnvawb7qsar2xshwwayi
       '@octokit/action': 3.18.1
       '@typescript-eslint/eslint-plugin': 5.30.0_5mtqsiui4sk53pmkx7i7ue45wm
       '@typescript-eslint/parser': 5.30.0_b5e7v2qnwxfo6hmiq56u52mz3e
@@ -3995,7 +3995,7 @@ packages:
       semver: 5.7.1
     dev: true
 
-  /@changesets/assemble-release-plan/5.1.3_t3ywtunilq53oqooixy7kemkoi:
+  /@changesets/assemble-release-plan/5.1.3_6bkrbdwcq4wpuel6jqtb3kupwi:
     resolution: {integrity: sha512-I+TTkUoqvxBEuDLoJfJYKDXIJ+nyiTbVJ8KGhpXEsLq4N/ms/AStSbouJwF2d/p3cB+RCPr5+gXh31GSN4kA7w==}
     dependencies:
       '@babel/runtime': 7.18.6
@@ -4023,13 +4023,13 @@ packages:
       - encoding
     dev: true
 
-  /@changesets/cli/2.23.0_kcozqtpxuwjzskw6zg5royevn4:
+  /@changesets/cli/2.23.0_kuwjnalnvawb7qsar2xshwwayi:
     resolution: {integrity: sha512-Gi3tMi0Vr6eNd8GX6q73tbOm9XOzGfuLEm4PYVeWG2neg5DlRGNOjYwrFULJ/An3N9MHtHn4r5h1Qvnju9Ijug==}
     hasBin: true
     dependencies:
       '@babel/runtime': 7.18.6
       '@changesets/apply-release-plan': 6.0.0
-      '@changesets/assemble-release-plan': 5.1.3_t3ywtunilq53oqooixy7kemkoi
+      '@changesets/assemble-release-plan': 5.1.3_6bkrbdwcq4wpuel6jqtb3kupwi
       '@changesets/changelog-git': 0.1.11
       '@changesets/config': 2.0.0
       '@changesets/errors': 0.1.4
@@ -4104,7 +4104,7 @@ packages:
     resolution: {integrity: sha512-5C1r4DcOjVxcCvPmXpymeyT6mdSTLCNiB2L+5uf19BRkDKndJdIQorH5Fe2XBR2nHUcZQFT+2TXDzCepat969w==}
     dependencies:
       '@babel/runtime': 7.18.6
-      '@changesets/assemble-release-plan': 5.1.3_t3ywtunilq53oqooixy7kemkoi
+      '@changesets/assemble-release-plan': 5.1.3_6bkrbdwcq4wpuel6jqtb3kupwi
       '@changesets/config': 2.0.0
       '@changesets/pre': 1.0.11
       '@changesets/read': 0.5.5

If you put these changes on macOS, the same error will be reported:

[BlackHole1]

I have created a reproduction repo for this issue: https://github.com/BlackHole1/test-pnpm-4961
This is basically where the problem code is located:

pnpm/packages/crypto.base32-hash/src/index.ts

Line 6 in b2d271d

return base32.stringify(crypto.createHash('md5').update(str).digest()).replace(/(=+)$/, '').toLowerCase()

See: https://github.com/BlackHole1/test-pnpm-4961/actions/runs/2595432720

macOS: https://github.com/BlackHole1/test-pnpm-4961/runs/7146689132?check_suite_focus=true#step:5:11

Linux: https://github.com/BlackHole1/test-pnpm-4961/runs/7146689233?check_suite_focus=true#step:5:11

Windows: https://github.com/BlackHole1/test-pnpm-4961/runs/7146689367?check_suite_focus=true#step:5:11

[BlackHole1]

When use: Buffer.from(str).toString('base64');, result diff:

So here's the problem:

pnpm/packages/crypto.base32-hash/src/index.ts

Line 10 in b2d271d

return createBase32Hash(await fs.promises.readFile(file, 'utf8'))

[BlackHole1]

When file content to buffer. Windows system has additional d0 hex.

I think it's a nodejs bug

[natemoo-re]

Oh wow, great investigation @BlackHole1! That's fascinating—never would have suspected this.

[zkochan]

@BlackHole1 good investigation. Could this be caused by different line endings on Windows?

[natemoo-re]

@zkochan @BlackHole1 Yes! Looks like this is caused by the line endings. Normalizing all \r\n to \n before creating the hash results in consistent behavior.

darwin result is: t3ywtunilq53oqooixy7kemkoi
linux result is: t3ywtunilq53oqooixy7kemkoi
win32 result is: t3ywtunilq53oqooixy7kemkoi

[BlackHole1]

Cool! It looks like we have a fix in place. 🌟🌟

[zkochan]

ok, I'll fix it soon

[natemoo-re]

Thanks for the quick turnaround, everyone! Just confirmed that this fixes it and we're unblocked! 🎉