Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pnpm 9 lockfile support #7993

Closed
1 task done
pawelblaszczyk5 opened this issue Apr 16, 2024 · 12 comments · Fixed by #7994
Closed
1 task done

pnpm 9 lockfile support #7993

pawelblaszczyk5 opened this issue Apr 16, 2024 · 12 comments · Fixed by #7994
Assignees

Comments

@pawelblaszczyk5
Copy link

Verify canary release

  • I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/pawelblaszczyk5/pnpm-prune-repro

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

1.13.3-canary.1

Describe the Bug

pnpm introduced a new lockfile format in the v9 major version. Formatting changes seems to mess up turbo prune, all stuff under packages key in output pnpm-lock.yaml is missing

Expected Behavior

turbo prune should properly prune new pnpm lockfile format

To Reproduce

  • Update pnpm to v9
  • Recreate lockfile so it utilizes the new 9.0 version
  • Try to run turbo prune some-package
  • Observe the prune output lockfile packages key being empty - it fails running pnpm i --frozen-lockfile because of that

Additional context

I've pushed to the reproduction repo the lockfile after pruning before and after updating to pnpm v9. You can check the difference between pnpm-lock-pruned-old.yaml and pnpm-lock-pruned-new.yaml

@pawelblaszczyk5 pawelblaszczyk5 added kind: bug Something isn't working needs: triage New issues get this label. Remove it after triage owned-by: turborepo labels Apr 16, 2024
@anthonyshew
Copy link
Contributor

Thanks for the report! It's expected that this would fail, given it's a new format, after all. We're aware of it and have it on our roadmap.

@anthonyshew anthonyshew changed the title pnpm@9.0.0 new lockfile format is incorrectly pruned pnpm 9 lockfile support Apr 16, 2024
@anthonyshew anthonyshew added area: prune turbo prune and removed needs: triage New issues get this label. Remove it after triage kind: bug Something isn't working labels Apr 16, 2024
@pawelblaszczyk5
Copy link
Author

Thanks for the report! It's expected that this would fail, given it's a new format, after all. We're aware of it and have it on our roadmap.

Yeah, I’m not surprised by this not working on day 1 of pnpm major release 😄 Just wanted to create this so it’s tracked somewhere and to make sure you’re aware of this. Thanks a lot!

@chris-olszewski chris-olszewski self-assigned this Apr 16, 2024
ddadaal added a commit to PKUHPC/SCOW that referenced this issue Apr 17, 2024
pnpm的最新版升级到了9.0,Lockfile版本从6.x更新到了9.x,存在不兼容问题。而现在在容器构建时,执行`pnpm
fetch`时下载的pnpm为最新版,而不是package.json中执行的版本,使得在执行pnpm
fetch时lockfile被更新到最新的9.x版本,但是后面在执行后续的pnpm指令时使用的pnpm版本为老版本,不兼容最新的lockfile。

由于 vercel/turbo#7993 问题,现在不能升级pnpm到最新的9.0版本。

此PR把package.json在执行`pnpm fetch`执行复制进了容器,使得执行`pnpm
fetch`时所使用的pnpm版本为指定的版本而非最新版,保证lockfile不会意外改变。
chris-olszewski added a commit that referenced this issue Apr 17, 2024
### Description

Closes #7993

Only thing that required updating is that as of `9.0.0-rc.0` the
lockfile version was updated to `9.0` instead of `7.0` that was used
from `9.0.0-alpha.0`-`9.0.0-beta.3`. I believe this was the only change
made since I added support for lockfile v7 in #7853.

### Testing Instructions

Added roundtrip test along with unit test for package resolution.

Manual test with repro provided in #7993
```
[0 olszewski@chriss-mbp] /tmp/pnpm-prune-repro $ turbo_dev --skip-infer prune app-a 
Generating pruned monorepo for app-a in /private/tmp/pnpm-prune-repro/out
 - Added app-a
 - Added pkg-a
 - Added tooling-config
[0 olszewski@chriss-mbp] /tmp/pnpm-prune-repro $ cd out 
[0 olszewski@chriss-mbp] /tmp/pnpm-prune-repro/out $ pnpm i --frozen-lockfile
Scope: all 4 workspace projects
Lockfile is up to date, resolution step is skipped
Packages: +2
++
Progress: resolved 2, reused 2, downloaded 0, added 2, done

devDependencies:
+ turbo 1.13.3-canary.1

Done in 245ms
```


Closes TURBO-2826
@chris-olszewski
Copy link
Contributor

This should be supported with 1.13.3-canary.2.

If anyone encounters any issues with pnpm 9, please reopen this issue along with a reproduction repo.

@VojtechVidra
Copy link

Hello, I'm still having issus with pnpm 9. You can try by running pnpm turbo prune --scope=web in the following branch: RBND-studio/flows-cloud#266

Hopefully it's not too complicated for you since it's not a minimal reproduction. Thanks!

@StuClift
Copy link

Using https://github.com/vercel/turbo/tree/main/examples/kitchen-sink

$ cd examples/kitchen-sink
$ rm pnpm-lock.yaml
$ corepack use pnpm@9.0.2
Installing pnpm@9.0.2 in the project...

Scope: all 10 workspace projects
Packages: +21 -21
+++++++++++++++++++++---------------------
Progress: resolved 1274, reused 1187, downloaded 0, added 0, done
Done in 2.1s
$ pnpm update turbo@1.13.3-canary.2
Packages: +3 -3
+++---
Progress: resolved 1274, reused 1187, downloaded 0, added 0, done

devDependencies:
- turbo 1.12.4
+ turbo 1.13.3-canary.2

Done in 2.8s
$ pnpm exec turbo prune admin
Generating pruned monorepo for admin in /Users/stuart.clift/Development/turbo/examples/kitchen-sink/out
 - Added @repo/eslint-config
 - Added @repo/jest-presets
 - Added @repo/typescript-config
 - Added @repo/ui
 - Added admin
  × error parsing dependency path: error Eof at: )(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-
  │ typescript@3.6.1(@typescript-eslint/parser@6.12.0(eslint@8.57.0)(typescript@5.3.3))(eslint-plugin-import@2.29.0)
  │ (eslint@8.57.0))(eslint@8.57.0)
  ╰─▶ error Eof at: )(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.6.1(@typescript-eslint/
      parser@6.12.0(eslint@8.57.0)(typescript@5.3.3))(eslint-plugin-import@2.29.0)(eslint@8.57.0))(eslint@8.57.0)
$ head -1 pnpm-lock.yaml
lockfileVersion: '9.0'
$ pnpm -v
9.0.2
$ turbo --version
1.13.3-canary.2

@schowdhuri
Copy link

Can confirm, I see the same error as @StuClift

Generating pruned monorepo for @app/api in /app/out
 - Added @app/api
  x error parsing dependency path: error Eof at: )(react@18.2.0)
  `-> error Eof at: )(react@18.2.0)
The command '/bin/sh -c turbo prune --scope=@app/api --docker' returned a non-zero code: 1

chris-olszewski added a commit that referenced this issue Apr 19, 2024
…ies (#8003)

### Description

Should address the reported issues in #7993 

I missed a change to the dependency path format where nested peer
dependencies are now shown e.g. if a has peer dependency b and b has
peer dependency c, then the resulting dependency path will be
`a@1.0.0(b@1.0.0(c@1.0.0))`.

Not sure when dependency path parsing got simplified, but I believe I
missed the host segment getting dropped which greatly simplifies the
format. The new parsing logic is a faithful port of the current JS
parsing code:
https://github.com/pnpm/pnpm/blob/main/packages/dependency-path/src/index.ts#L91

### Testing Instructions

Added unit test for parsing a dependency path that contains a nested
peer dependency. Existing test suite.


Closes TURBO-2848
@chris-olszewski
Copy link
Contributor

#8003 was released inv1.13.3-canary.3 and should address issues with parsing dependencies with nested peer dependencies.

@jordanebelanger
Copy link

jordanebelanger commented Apr 25, 2024

@chris-olszewski Any eta on how long this will be out of canary, I am a bit hesitant to pin turbo or pnpm in a bunch of place in my project if this is coming soon.

@chris-olszewski
Copy link
Contributor

@jordanebelanger 1.13.3 just got released this morning

@thexpand
Copy link

thexpand commented Apr 29, 2024

I'm still getting a warning when I build on Vercel, even using the latest (1.13.3) turbo version in my turbo repo:

Detected `pnpm-lock.yaml` version 9 generated by pnpm 8
Running "install" command: `pnpm install`...
Scope: all 8 workspace projects

WARN  Ignoring not compatible lockfile at /vercel/path0/pnpm-lock.yaml

@rajatkapoor
Copy link

I'm still getting a warning when I build on Vercel, even using the latest (1.13.3) turbo version in my turbo repo:

Detected `pnpm-lock.yaml` version 9 generated by pnpm 8
Running "install" command: `pnpm install`...
Scope: all 8 workspace projects

WARN  Ignoring not compatible lockfile at /vercel/path0/pnpm-lock.yaml

@thexpand This workaround helped me

Setting ENABLE_EXPERIMENTAL_COREPACK=1 in the env helps me get around this issue if you have set

  "packageManager": "pnpm@9.0.6"

in your package.json

You can follow the steps as listed here: https://vercel.com/docs/deployments/configure-a-build#corepack

@pawelblaszczyk5
Copy link
Author

I verified and it seems to work correctly with latest turbo version, closing the issue, tyvm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants