[rush-lib] Fix Set ignore-compatibility-db=true when rush installation

[sherlockfeng]

Background
When using different versions of pnpm to install in TTFE Monorepo. There are unwanted package changes in lockfile. The reason is that pnpm bundled @yarnpkg/package-extensions, which mysteriously installs unrelated packages.

Related issue:

PNPM mysteriously installs unrelated packages · Issue #5132 · pnpm/pnpm (github.com)

Zoltan, added a new "ignore-compatibility-db" settings to disable this behaviour 6.34.0 and 7.9.0-0

For Rush.js side, we do the following things:

"rush install/update" should always set "ignore-compatibility-db=true". It's unconditional because Rush's recommended way to use compatibility db would be to copy+paste the settings to "pnpm.packageExtensions" or "pnpmfile.cjs". (The rules must be stored in Git rather than installed via NPM).
Rush should print some kind of warning if the rush.json pnpmVersion specifies a version affected by this problem.

The affected versions are ">=6.32.12" and >= 7.0.1, and then < 6.34.0 and < 7.9.0-0 version

The message will be like: Warning: Your rush.json specifies a pnpmVersion with a known issue that may cause unintended version selections. It's recommended to upgrade to PNPM >=6.34.0 or >=7.9.0. For details see: https://rushjs.io/link/pnpm-issue-5132

[ghost]

All CLA requirements met.

[TheBit]

Hello! @sherlockfeng, @octogonz
Did you try it?
I was just working on a PR in my project to update Rush and pnpm to the latest, and got Unknown option: 'ignore-compatibility-db' on rush update:

[octogonz]

This is a regression, thanks for reporting!
As a workaround, use PNPM 7.9.0-0 or else Rush 5.75.0

[sherlockfeng]

i found that ignore-compatibility-db only supported in .npmrc, i will fix it today~

[sherlockfeng]

@TheBit @octogonz fixed in 3578

[octogonz]

🚀 This fix was released with @microsoft/rush 5.76.1

[TheBit]

@octogonz Hi, any guesses?
I can't commit with 5.76.1 (also no node_modules in common/temp/install-run) due to the usage of install-run-rush.js lint-staged in the pre-commit hook which fails on Installing @microsoft/rush... code ETARGET... notarget No matching version found for @microsoft/rush@5.76.1..

I also noticed the lock file gets changed after rush purge and rush update --full (the changes inside are mostly about removing transitivePeerDependencies like bluebird and supports-color)

The lock file is not changed and I can commit with previous versions, e.g. 5.74.0 and 5.75.0.

P.S. I also did npm update -g @microsoft/rush and restarted the terminal (and I use workspaces and strict peers)

[TheBit]

Inspected 3 tarballs from npm (5.75.0, 5.76.0, 5.76.1) - couldn't find anything suspicious (well, at least from my point of view)

[octogonz]

I can't commit with 5.76.1 (also no node_modules in common/temp/install-run) due to the usage of install-run-rush.js lint-staged in the pre-commit hook which fails on Installing @microsoft/rush... code ETARGET... notarget No matching version found for @microsoft/rush@5.76.1..

@TheBit After an NPM package gets published, npmjs.com's CouchDB sometimes takes a little while to sync and make the version available. Also, if your company is using a caching NPM proxy, it may take some time for the proxy to pick up the new version.

It seems to be installing fine for me, for example: PR #3580

I also noticed the lock file gets changed after rush purge and rush update --full (the changes inside are mostly about removing transitivePeerDependencies like bluebird and supports-color)

This is by design:

rush/CHANGELOG.md

5.76.0

• "rush install/update" should always set ignore-compatibility-db=true and print warning if the rush.json pnpmVersion specifies a version affected by this problem.

Details in this GitHub issue: PNPM mysteriously installs unrelated packages · Issue #5132 · pnpm/pnpm (github.com)

In a nutshell, if you are using PNPM >= 6.32.12 then PNPM applies hidden version fixups (from a table published as @yarnpkg/extensions). These fixups are helpful for beginners, but problematic for a large Rush repo because (1) they are invisible to people who are trying to troubleshoot problems using pnpmfile.cjs, and (2) the fixups vary depending on the PNPM version number, which causes mysterious lockfile churn when upgrading/downgrading the package manager.

Rush 5.76.0 disables this behavior. Disabling it will cause lockfile churn for people whose lockfiles were influenced by those fixups. The workaround would be to use pnpmfile.cjs to explicitly re-add the relevant @yarnpkg/extensions rules from your release of PNPM. You can find these rules in pnpm/dist/pnpm.cjs (screenshot).

We should call this out more clearly and provide better docs. I'll do that tonight.

[TheBit]

@octogonz thanx! It works now. It looks like I was rushing too much with the Rush ;)