Cannot delete an optional dependency by using .pnpmfile.cjs - becomes a dependency instead?

[EdwardDrapkin]

Verify latest release

• I verified that the issue exists in the latest pnpm release

pnpm version

8.13.0

Which area(s) of pnpm are affected? (leave empty if unsure)

Dependencies resolver

Link to the code that reproduces this issue or a replay of the bug

No response

Reproduction steps

I'm trying to prevent an optional dependency of a transitive dependency from being installed. Adding the following pnpmfile:

module.exports = {
  hooks: {
    readPackage: (pkg) => {
      if(pkg.name === 'ssh2') {
        delete pkg.optionalDependencies['cpu-features'];
        delete pkg.dependencies['cpu-features'];
        console.log(pkg);
      }

      return pkg;
    }
  }
}

and the dependency ssh2-sftp-client will re-create the issue.

Describe the Bug

The package ssh2-sftp-client has a dependency on ssh2. ssh2 has an optionalDependency on cpu-features which I specifically want to exclude from installation. I am trying to exclude it using a .pnpmfile.cjs that looks like this:

module.exports = {
  hooks: {
    readPackage: (pkg) => {
      if(pkg.name === 'ssh2') {
        delete pkg.optionalDependencies['cpu-features'];
        delete pkg.dependencies['cpu-features'];
        console.log(pkg);
      }

      return pkg;
    }
  }
}

The output of running pnpm i includes the modified package.json, which looks as I'd expect:

{
  dependencies: { asn1: '^0.2.6', 'bcrypt-pbkdf': '^1.0.2' },
  engines: { node: '>=10.16.0' },
  name: 'ssh2',
  optionalDependencies: { nan: '^2.17.0' },
  scripts: {
    install: 'node install.js',
    rebuild: 'node install.js',
    test: 'node test/test.js',
    lint: 'eslint --cache --report-unused-disable-directives --ext=.js .eslintrc.js examples lib test',
    'lint:fix': 'npm run lint -- --fix'
  },
  version: '1.14.0',
  devDependencies: {},
  peerDependencies: {}

The output lock file has not just removed cpu-features from the optionalDependencies, it's added it to the dependencies as well!

  /ssh2@1.14.0:
    resolution: {integrity: sha512-AqzD1UCqit8tbOKoj6ztDDi1ffJZ2rV2SwlgrVVrHPkV5vWqGJOVp5pmtj18PunkPJAuKQsnInyKV+/Nb2bUnA==}
    engines: {node: '>=10.16.0'}
    requiresBuild: true
    dependencies:
      asn1: 0.2.6
      bcrypt-pbkdf: 1.0.2
      cpu-features: 0.0.9
    optionalDependencies:
      nan: 2.18.0
    dev: false

Expected Behavior

cpu-features is not installed

Which Node.js version are you using?

v18.19.0

Which operating systems have you used?

• macOS
• Windows
• Linux

If your OS is a Linux based, which one it is? (Include the version if relevant)

No response

[KSXGitHub]

The lockfile will always include optional dependencies regardless of whether pnpm installs them.

You should run ls node_modules/.pnpm/cpu-features* to verify whether pnpm has actually installed it.

[KSXGitHub]

Hello again, @EdwardDrapkin. This is a non-issue and should be closed, but I have left this issue open to wait for your response. Have you received my message?

[EdwardDrapkin]

PNPM definitely installs it - bun hard crashes when that package is imported, it's conditionally imported by an upstream dependency, and removing it from node_modules allows bun to run the app I'm trying to run.

Even if it didn't, if it's removed from the optional dependencies, it should not be added to the lockfile as a hard dependency... a user who goes out of their way to remove something from an optional dependency should not suggest to the package manager that they mean to make some package that was previously optional now mandatory. This is an issue in both the lockfile generated and that the dependency gets installed.

If you read the code I pasted and the lockfile output, you'll see that the pnpmfile is not being correctly applied at all. That dependency is manually deleted from both optionalDependencies and dependencies, and no permutation of option actually removes it from the lockfile or the node_modules directory.

[KSXGitHub]

PNPM definitely installs it

Please do a clean install again and run ls node_modules/.pnpm/cpu-features* then tell me the output.

The command I use was pnpm install --no-optional ssh2 && ls node_modules/.pnpm/cpu-features* which errors for the second command, proving that optional are not installed.

When I try to reproduce your steps (that is: create a .pnpmfile.cjs with your content then run pnpm add ssh2-sftp-client), the lockfile doesn't even have cpu-features (the output of cat pnpm-lock.yaml | grep cpu is empty). So you must have made some mistakes or there were something else you haven't told me.

[jfirebaugh]

I think there is a real bug in pnpm with respect to .pnpmfile.cjs and existing lockfile / node_node modules. Here are my repro steps.

1. Start with the following package.json and no pnpm-lock.yaml:

   {
     "dependencies": {
       "piscina": "^4.3.1"
     }
   }
   

2. Run pnpm install.
3. Add the following .pnpmfile.cjs:

   function readPackage(pkg, context) {
     if (pkg.name === 'piscina') {
       // Remove an optional dependency that does not build under Bazel.
       delete pkg.optionalDependencies['nice-napi']
     }
   
     return pkg
   }
   
   module.exports = {
     hooks: {
       readPackage
     }
   }
   

4. Run pnpm install again.

Expected result: pnpm removes the optional dependency.
Actual result: pnpm prints "Lockfile is up to date, resolution step is skipped / Already up to date" and the lockfile and node_modules contents are unchanged.

To get to the expected state, you have to run rm -rf pnpm-lock.yaml node_modules && pnpm install. Deleting just the lockfile or just node_modules is insufficient.

[zkochan]

pnpm update --depth Infinity would also remove the optional dependency.

The issue is that pnpm doesn't know when .pnpmfile.cjs was changed and doesn't know that it should reanalyze the whole lockfile again.

In any case, I think we should support a new setting for selectively ignoring some optional dependencies. Something like, package.json:

{
  "pnpm": {
    "ignoredOptionalDependencies": ["nice-napi"]
  }
}

[jfirebaugh]

Should pnpm write a checksum for .pnpmfile.cjs into the lockfile, like it does for package extensions?

In any case, I think we should support a new setting for selectively ignoring some optional dependencies.

That would work for me too.

[zkochan]

New feature: #7714

[zkochan]

I believe the original issue has been fixed by this PR: #7704