pnpm.fetchDeps: init

[Scrumplex]

Description of changes

Nowadays pnpm is almost unavoidable when working with any larger Node.js project. In an effort to support this package manager in Nixpkgs, this PR introduces pnpm.fetchDeps which produces a FOD containing a pnpm store that can be used for a reproducible offline build.

Closes #231513

To-dos

• Disable withNode for pnpm when used in a derivation
• Add a pnpm server start script to pnpmDeps FODs
• Support pnpm-workspace.yaml projects

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[getchoo]

this could be a near drop-in replacement for methods used in geph, kiwitalk, pot, vesktop, and youtube-music

edit: can also be used in gitbutler and modrinth-app

[Scrumplex]

I got rid of the supportedArchitectures code, as --force achieves the same without having to specify all platforms

[nyabinary]

Needs a rebased :3

[doronbehar]

I can confirm that kiwitalk is working fine. Thanks.

But bash-language-server doesn't seem to work.

Aish I didn't mean to push that, thanks for noticing it quickly and for reviewing.

[doronbehar]

It seems no body has anything further to add / review. Merging 🎉.

[SuperSandro2000]

We should have put the overrides for pnpm for the consuming packages inside the packages. pnpm is really bitchy about a wrong version and that needs to be closely tracked.

Also maybe we can add some small script, to detect this and catch this error early.

youtube-music-pnpm-deps> Running phase: installPhase
youtube-music-pnpm-deps>  WARN  using --force I sure hope you know what you are doing
youtube-music-pnpm-deps>  WARN  Ignoring not compatible lockfile at /build/source/pnpm-lock.yaml
youtube-music-pnpm-deps>  ERR_PNPM_LOCKFILE_CONFIG_MISMATCH  Cannot proceed with the frozen installation. The current "overrides" configuration doesn't match the value found in the lockfile
youtube-music-pnpm-deps>
youtube-music-pnpm-deps> Update your lockfile using "pnpm install --no-frozen-lockfile"

[doronbehar]

We should have put the overrides for pnpm for the consuming packages inside the packages.

I don't understand this sentence...

[SuperSandro2000]

They are right now in all-packages where you can easily miss them and wonder why your pnpmDeps build is failing for no apparent reason, since the error message is so bad.

see #317739 for my idea on how to improve that

[NyCodeGHG]

I'm trying to package renovate using pnpm.fetchDeps, but I get the error ERR_PNPM_NO_OFFLINE_META when executing pnpm deploy, looks like this is the same problem as in #309100

I can only reproduce it inside the nix sandbox, outside of it (and in a network namespace without internet access), it works fine, not sure how to debug this further.

build log:
https://gist.github.com/NyCodeGHG/073f40768d21796fac69cd98b8e0a8df

[GetPsyched]

Dropping in here to ask about monorepo/workspaces support. I'm packaging a monorepo project and noticed from the discussion here that it isn't supported yet.

Is there an issue or, better yet, a draft PR that I can track for this?

[pluiedev]

Dropping in here to ask about monorepo/workspaces support. I'm packaging a monorepo project and noticed from the discussion here that it isn't supported yet.

Is there an issue or, better yet, a draft PR that I can track for this?

I believe you're looking for #316908

[doronbehar]

I'm trying to package renovate using pnpm.fetchDeps, but I get the error ERR_PNPM_NO_OFFLINE_META when executing pnpm deploy, looks like this is the same problem as in #309100

@NyCodeGHG since the issue #309100 was diagnosed as a workspaces issue, perhaps your issue with renovate is the same? See:

• pnpm.configHook / npmHooks.npm{Install,Build}Hook: Support pnpm-workspace.yaml projects #316908

[NyCodeGHG]

I'm not sure if thats the problem, renovate has a pnpm-workspace.yaml file without a single member
don't ask me why

[doronbehar]

I'm not sure if thats the problem, renovate has a pnpm-workspace.yaml file without a single member don't ask me why

Does it help to simply remove it? :)

[NyCodeGHG]

i just tried, no.
looks like they have that file to prevent pnpm from also installing dependencies from test fixtures