PNPM is incorrectly providing the same dependency version to multiple packages relying on different versions

[JamesSeymour]

pnpm version: 7.8.0

Code to reproduce the issue:

Attached is a very basic project with two packages, each relying on only a single devDependency (jest) - but each package has a different version of that same dependency. I have added a script jest-version which just logs the jest version for ease of use.

pnpm-dependency-issue.zip

Package-1 - package.json:

  "name": "package-1",
  "version": "1.0.0",
  "description": "Package 1",
  "license": "UNLICENSED",
  "scripts": {
    "jest-version": "jest --version"
  },
  "devDependencies": {
    "jest": "^27.5.1"
  }
}

Package-2 - package.json:

  "name": "package-2",
  "version": "1.0.0",
  "description": "Package 2",
  "license": "UNLICENSED",
  "scripts": {
    "jest-version": "jest --version"
  },
  "devDependencies": {
    "jest": "^24.8.0"
  }
}

Expected behaviour:

I would expect when running pnpm jest-version in package-1 I would get an output of 27.5.1, and when running pnpm jest-version in package-2 I would get an output of 24.8.0.

Actual behaviour:

I get an output of 24.8.0 for the command in both packages.

Additional information:

• node -v prints: v16.16.0
• Windows, macOS, or Linux?: Windows and macOS.

For what its worth, this does behave expected on npm (8.11.0).

[VividLemon]

I'm not a contributor or anything, just curious, but this seems to be only a "checking command version" problem, the actual packages they're using are different

[JamesSeymour]

I can also see that the packages look correct in the node_modules. But when a command is run, it uses the incorrect package. For example I just added a basic test to package-1 and added a standard "test": "jest" to the scripts in the package.json. I then went into the root of the project and added a console log (logging the version) to where the tests are ran internally in jest in the different versions:

node_modules/.pnpm/@jest+core@24.9.0/node_modules/jest-runner/build/runTest.js
and
node_modules/.pnpm/@jest+core@27.5.1/node_modules/jest-runner/build/runTest.js

When running pnpm test in package-1 it logs 24.9.0 - not what I would expect.

[grimnomad]

Hey, I have a similiar problem. Running jest --showConfig or jest --version will output an other version than pnpm why jest.

Running pnpm -v prints 7.3.0.

[zkochan]

Looks like it is some issue with Jest, when hoisting is used.

Add this .npmrc to the root of your workspace:

hoist=false

Run pnpm install and it works fine.

[palexandrefernandes]

I'm unfortunately experiencing a similar problematic behaviour. I don't think this is an issue with just Jest and hoisting in particular given that this used to work properly before I upgraded to pnpm 7 (in version 6.32.8).

Everything worked as expected before updating, the correct version of Jest was instanced in the expected workspace, but not anymore.

One thing that I noticed is that (in version 7) this only happens after I run pnpm install. If I reinstall Jest manually by uninstalling and reinstalling it (with pnpm i -D jest@version), the next time I run pnpm jest --version it will print the correct version. But as soon as I run pnpm installagain, the wrong version will be printed again.

[havenchyk]

in v6 in packages/A I run pnpm jest --version and see the version from packages/A/package.json - 28, with pnpm 7 I do the same and get another version - 26.6.3 (which is probably from packages/B package.json)

I checked packages/A/node_modules/.bin/jest and found out there

exec node "$basedir/../../../../nodemodules/.pnpm/jest@28.1.0@types+node@14.17.21/node_modules/jest/bin/jest.js" "$@"
which is correct 🙂 and if I run this command manually in my terminal from the packages/A/node_modules/.bin folder, it works as expected - prints version 28

the main diff I found is in this line export NODE_PATH="/Users/uhavenchyk/p/my-project/node_modules/.pnpm/node_modules" from the packages/A/node_modules/.bin/jest in v7, in v6 it's a way longer

[ConcernedHobbit]

This seems to also happen with tsc and multiple @types library versions, with tsc always using the alphabetically first (older) @types library.

[zkochan]

@ConcernedHobbit does this help with the types issue #5176 (comment)?

[zkochan]

Looks like it works if I remove NODE_PATH from the command shim... And all tests seem to pass. But I am hesitant. It used to be required.

#5290

[WhiteKiwi]

I am using the public-hoist-pattern option in Monorepo as follows.

public-hoist-pattern[]='*webpack*'
public-hoist-pattern[]='*babel*'
public-hoist-pattern[]='*eslint*'
public-hoist-pattern[]=buffer
public-hoist-pattern[]=script-loader

hoist=false solves the jest issue, but I need to install the above libraries on each packages in monorepo. 😭
Is there a good way to solve it without hoist=false?

My team is in the process of pnpm 7 migration today.
This problem is expected to be a big issue in the use of monorepo at pnpm7. 🙇

I was mistaken.
Even if hoist=false, public-hoist-pattern appears to work.
hoist=false works to me.

[maschwenk]

I highly recommend turning off public-hoist-pattern[] completely. We had it on for years with a large monorepo and it caused us unbelievable issues because we had phantom @types dependencies that would result in very confusing and non-deterministic typechecking results.

[WhiteKiwi]

Thanks for the comment. we were forced to use it because it is legacy. We cleared all the public-hoist-pattern today. 👍

[zackerydev]

If anyone else is hitting this we have a sort of workaround that doesn't require changing hoist patterns

"jest": "./node_modules/jest/bin/jest.js --colors",
 "test": "./node_modules/jest/bin/jest.js --colors",

This resolves to the right version of jest

[ConcernedHobbit]

@ConcernedHobbit does this help with the types issue #5176 (comment)?

Sure, this helps with jest and type issues, but then there's problems with developing component libraries using peerDependencies for react, as then they aren't installed. This then requires configuring public-hoist-pattern for all dependencies that might be necessary. Again, an exclusive public-hoist-pattern-exclude option would be a lifesaver. Now I'm spending more time manually playing whack-a-mole with packages that should be hoisted than necessary.

[ConcernedHobbit]

Looks like it works if I remove NODE_PATH from the command shim... And all tests seem to pass. But I am hesitant. It used to be required.

#5290

This would probably be the best fix, as I would expect jest with multiple versions to work natively in pnpm. The fact that it doesn't and configuration tricks are required is a bit disappointing :/ Any updates? Seems like it's stuck on some sort of Next issue.

[ConcernedHobbit]

@ConcernedHobbit does this help with the types issue #5176 (comment)?

Dang, @types/react is breaking again. The weird error message may be a clue to what's going wrong. To recap, the project has two packages, one of which uses @types/react@18 and one uses @types/react@17. With any combination of hoist=false, public-hoist-pattern=[], public-hoist-pattern[]=typesetc.,tsc` fails with a weird import:

(this is in the package which has @types/react@^17)

Type 'React.ReactNode' is not assignable to type 'import("/home/[redacted-path-to-root-folder]/node_modules/.pnpm/@types+react@18.0.21/node_modules/@types/react/index").ReactNode'.

This might be a tsc issue, but I would still assume that it should work with pnpm out-of-the-box.

[boiboif]

@ConcernedHobbit #5176 (comment)
I have the same problem. Have you solved this problem?

[hahanein]

@boiboif @ConcernedHobbit This issue cost us several days. But in our case it was not an issue with pnpm but with tsc and ts-jest.

More specifically, in certain scenarios tsc traverses up your directory tree. Effectively ignoring whatever you declared in "typeRoots".

To fix this we had to add "react" to "paths" in tsconfig.json. See:

• typeRoots is not resolved as part of compilation microsoft/TypeScript#27026 (comment)
• https://stackoverflow.com/questions/52463716/typescript-cannot-import-self-in-types-override-in-other-types-root-ts7016/52470989#52470989

[asprouse]

@zkochan Is there a reason extend-node-path was removed? Before upgrading to v7 I used this to workaround the issue. Now the suggested fix is to set hoist=false. We have always used the default hoist settings and I always thought pnpm was strict by default when it came to dependencies. What are the downsides to hoist=false?

[asprouse]

I am trying to get PNPM to work with NX and this issue breaks distributed task execution. Since NX uses PNPM to run package.json scripts shimming the node path leads to wrong versions being used. I tried hoist=false but that breaks NX 🤦. Restoring the extend-node-path seems like a good option if we cannot merge #5290. I made a PR to restore this option. #5910

[zkochan]

I tried hoist=false but that breaks NX 🤦.

Instead of hoist=false you can try to specifically hoist dependencies that nx cannot find by setting hoist-pattern. Or you can you package extensions

What are the downsides to hoist=false?

The only downside is that some packages are published with require statements to other packages that are not added to their dependencies. These package will fail with hoist=false. You can selectively hoist the packages that appear in the errors or use package extensions to add the missing dependencies to the packages that use them.

[iobuhov]

Have same problem on 7.27.0

❯ pnpm -v
7.27.0

❯ pnpm ls --depth 0 jest
Legend: production dependency, optional only, dev only

datagrid-number-filter-web@2.3.1

devDependencies:
jest 29.4.2

❯ pnpm exec jest --version
26.6.3

[zkochan]

The current workaround for this issue is to set extra-node-path=false in an .npmrc file in the root of your workspace. Then remove node_modules and reinstall it.

[zkochan]

I cannot reproduce the next dev issue anymore. And I cannot think of a reason why anything would break if I remove NODE_PATH patching. So, I will remove it

[levrik]

@zkochan This change suddenly broke all of our CI builds with GraphQL Codegen packages. Manually setting extend-node-path=true in our .npmrc seems to fix it. Setting public-hoist-pattern[]=*@graphql-codegen* also does it.

[zkochan]

I have downgraded latest.

Create a repo that reproduces the issue if you can.

[levrik]

@zkochan Still trying to figure out if this change in PNPM maybe also just made a configuration mistake on our side apparent that just worked by accident so far. Will come back with results soon.

[iobuhov]

+1
Event if you have hoist=* module resolution is not working same way when update to 7.29.
Adding "phantom" packages to public-hoist-pattern helps, but because it's minor update it probably should not break current setup.

Please never mind, found another thread.

[levrik]

@zkochan Okay. Figured out that in my case it was never intended to work from the GraphQL Codegen side and just happened to work by accident in Yarn and PNPM until now. To cite from dotansimha/graphql-code-generator#3275 (comment):

The basic instruction is to install the plugins you are using under plugins: section.

We previously used a plugin in that config section which wasn't installed as a direct dependency. The change in PNPM here broke it now but that's probably fine.

But from looking at the issue linked by @iobuhov there seem to be more tools which required sub-dependencies to be discoverable which this change apparently broke.

[iobuhov]

I agree with @levrik that pnpm not "broke" things. It's just some weird config changes. I see that many packages that were built using node module resolution system, including beble for example, don't play nice with pnpm isolated system as they rely on default resolution strategy and because of that not always list their deps explicitly.
But what was unexpected is to see this switch to more strict approach between 7.28.0 and 7.29.2. I think this should be an opt-in for the end user. But again, this is life and bug happens.

[zkochan]

Right, I was not expecting that the removal of NODE_PATH would break anything as the node_modules/.pnpm/node_modules directory should be looked into by default, when resolving require statements in subdirs of node_modules/.pnpm/. Maybe some pluggable tools use the value of NODE_PATH explicitly.

[zkochan]

🚢 7.29.3

[phoenixeliot]

I have this in my package.json to work around a Jest issue with snapshots not working with prettier v3:

    "prettier": "^3.0.0",
    "prettier-2": "npm:prettier@^2",

It seems that, when I do pnpm install, it nondeterministically decides which one gets to be the one that runs when I do pnpm exec prettier --version. Removing one of the two to force the other to be the only installed, reinstalling, then putting it back in package.json, reinstalling, seems to result in a random outcome as to which one is the one used for pnpm exec prettier.

I guess I'll modify my package.json script to use the exact node_modules path to prettier and use that instead.

I added this to my scripts section, seems to work:

"prettier": "node_modules/prettier/bin/prettier.cjs",