Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI complains of outdated lockfile but says the lockfile is up to date #5144

Closed
tleunen opened this issue Aug 2, 2022 · 45 comments · Fixed by #5307
Closed

CI complains of outdated lockfile but says the lockfile is up to date #5144

tleunen opened this issue Aug 2, 2022 · 45 comments · Fixed by #5307
Assignees
Milestone

Comments

@tleunen
Copy link

tleunen commented Aug 2, 2022

The CLI says the lockfile is up to date but it still failing when it runs on CI. Running pnpm i locally doesn't create any updates to the lockfile.

pnpm version: 7.8.0

The issue exists since the release of 7.6.0. It works fine in 7.5.2

Expected behavior:

CI doesn't complain about outdated file when the lockfile is updated.

Actual behavior:

CI complains the lockfile is outdated

pnpm install --prefer-offline

Scope: all 21 workspace projects
Lockfile is up-to-date, resolution step is skipped
 ERR_PNPM_OUTDATED_LOCKFILE  Cannot install with "frozen-lockfile" because pnpm-lock.yaml is not up-to-date with packages/eslint-plugin/package.json

Note that in CI environments this setting is true by default. If you still need to run install in such cases, use "pnpm install --no-frozen-lockfile"

Exited with code exit status 1

Additional information:

  • node -v prints: 16.16.0
  • Windows, macOS, or Linux?: OSX/Linux
@zkochan
Copy link
Member

zkochan commented Aug 2, 2022

can you create repository that reproduces the issue?

@tleunen
Copy link
Author

tleunen commented Aug 3, 2022

I'll try to make a repro of this, because this is happening in our monorepo at work.

@diesal11
Copy link

diesal11 commented Aug 4, 2022

I'm finding a the same issue which i also fixed by downgrading to 7.5.2, it seems that Pnpm wants to generate a different lockfile on Macos vs Linux?

Unfortunately this is also happening on my work monorepo so can't share a repro either.

@zkochan
Copy link
Member

zkochan commented Aug 6, 2022

Do you use the auto-install-peers=true setting?

@tleunen
Copy link
Author

tleunen commented Aug 6, 2022

Exactly, we have this setting enabled at the moment. I'll be able to try on Tuesday with the option set to false instead to see if it makes a difference

@zkochan
Copy link
Member

zkochan commented Aug 6, 2022

It was caused by this change: #5067

And a similar issue was already reported and fixed: #5080

But looks like there are still some edge cases.

@supalarry
Copy link

supalarry commented Aug 8, 2022

Same error here when trying to upgrade to pnpm 7.9.0 with auto-install-peers being true. 7.5.2 works without problems.

@zkochan
Copy link
Member

zkochan commented Aug 9, 2022

Have you tried to remove your lockfile and node_modules, then to generate again and commit it? The issue is that pnpm now installs the root peer dependency automatically, so if you did not have the peer dependency in your package.json, your lockfile will now be considered outdated.

@tleunen
Copy link
Author

tleunen commented Aug 10, 2022

Have you tried to remove your lockfile and node_modules, then to generate again and commit it?

Unfortunately this doesn't fix the issue.

The only way to fix it seems to be by removing auto-install-peers

@supalarry
Copy link

supalarry commented Aug 17, 2022

@zkochan removing lockfile and node_modules did not work. What helps is moving a peer dependency from peerDependencies to devDependencies. For example I had following error

ERR_PNPM_OUTDATED_LOCKFILE  Cannot install with "frozen-lockfile" because pnpm-lock.yaml is not up to date with config/stylelint-config/package.json

and stylelint-config/package.json had autoprefixer in peerDependencies and once I moved it to devDependencies the error disappeared. But I have more cases like this and unless I am missing something it does not make sense for a peer dependency to be moved to dev dependencies

pnpm at version 7.9.2

@supalarry
Copy link

@tleunen same here, once I set auto-install-peers to false the error disappeared.

@jason-ha
Copy link

Failure appears to happen when the peer dependency is a devDependency in root or any other workspace.
Snippet of pnpm-lock.yaml when issue occurs using file set listed below:

  packages/api-extractor-configuration:
    specifiers:
      '@microsoft/api-extractor': 7.x
    devDependencies:      <----- This would be "dependencies:" when there is no issue.
      '@microsoft/api-extractor': 7.29.5

Here is a minimalish set of files:
package.json:

{
  "name": "@my-stuff/pnpm-peer-lock-failure",
  "private": true,
  "devDpendencies": {
    "@microsoft/api-extractor": "^7.29.0"
  }
}

pnpm-workspaces.yaml:

prefer-workspace-packages: true
packages:
  - 'packages/*'

packages\api-extractor-configuration\package.json:

{
  "name": "@my-stuff/api-extractor-configuration",
  "version": "0.0.0-placeholder",
  "peerDependencies": {
    "@microsoft/api-extractor": "7.x"
  }
}

I attempted workaround of making the few peers we have be regular dependencies in the root, but the issue remained so long as any other package had the peer as a devDependency.

@grug
Copy link

grug commented Sep 2, 2022

I'm also experiencing this problem - unfortunately it's a massive project so we can't just move dependencies around until everything lines up nicely. Looking forward to seeing a resolution to this issue.

@jason-ha
Copy link

jason-ha commented Sep 2, 2022

For one repository I went ahead and added all of the peer dependencies explicitly. For another that we are upgrading from pnpm 6 to 7, we will be setting strict-peer-dependencies=false until there is a fix.

@zkochan
Copy link
Member

zkochan commented Sep 4, 2022

@jason-ha thanks

I was able to reproduce it https://github.com/zkochan/pnpm-issue-5144

@zkochan zkochan self-assigned this Sep 5, 2022
@zkochan zkochan added this to the v7.11 milestone Sep 5, 2022
zkochan added a commit that referenced this issue Sep 5, 2022
zkochan added a commit that referenced this issue Sep 5, 2022
* fix: auto installing peer dep in a workspace

close #5144

* fix: auto installing peer dep in a workspace

* fix: auto installing peer dep in a workspace
@grug
Copy link

grug commented Sep 5, 2022

@zkochan I see you've closed this - is there a resolution to this?

@zkochan
Copy link
Member

zkochan commented Sep 5, 2022

Soon in v7.11.0

@grug
Copy link

grug commented Sep 5, 2022

Amazing - really appreciate it. I know it's probably difficult to say, but do you have an idea of when that will be published? Are we talking days/weeks/months? I'm assuming pretty soon 😄

@zkochan
Copy link
Member

zkochan commented Sep 5, 2022

Soon means like 5 minutes

https://github.com/pnpm/pnpm/actions/runs/2992217063

@grug
Copy link

grug commented Sep 5, 2022

Hahaha you're a legend!

@grug
Copy link

grug commented Sep 5, 2022

I'm still receiving this issue on 7.11.0 but I'm not 100% sure how I'm going to be able to reproduce this for you as it's happening in a work project repository and it's pretty big.

@zkochan
Copy link
Member

zkochan commented Sep 5, 2022

Have you removed node_modules and pnpm-lock.yaml and create a new lockfile?

@emahuni
Copy link

emahuni commented Sep 6, 2022

I just love the fact that it installs peers and transitive peer dependencies as well. Yarn 1 was my go to pm, but for years they've been adamantly refusing to work on this seemingly intuitive feature. I had to use other means to auto install peers for years hoping they'd fix this. I ditched it coz yarn 2+ is actually removing more features or making it harder to use, meaning we have to use other workarounds. EG, Try yarn link in v1, works well, but not in yarn 3, I don't even know how to do it in yarn 3, it was just frustrating that I had to just leave it, yet linking is supposed to be so simple. PNPM is fast and does everything in a much much better and intuitive way, and seem to be more supportive of how people use PM's; @zkochan and team listen to feedback and they actually made this with regards to how people use package managers not how they want people to use package managers. Thanks a million, we will support your efforts.

@yugasun
Copy link

yugasun commented Sep 8, 2022

After I update pnpm to 7.11.0 and set auto-install-peers=true, the issue never came up again.

@zkochan
Copy link
Member

zkochan commented Sep 8, 2022

yep, got this and had to delete pnpm entire store, my lock files (committed the deletion into vcs, I did this initially without these other measures and it didn't work), node_modules, .pnp.cjs; as if it's a new machine for the project.

@emahuni you had to only remove the lockfile and node_modules. The reason you need to remove node_modules is because there is a duplicate of the lockfile inside node_modules/.pnpm/lock.yaml. The store should not be removed.

@emahuni
Copy link

emahuni commented Sep 12, 2022

yep, got this and had to delete pnpm entire store, my lock files (committed the deletion into vcs, I did this initially without these other measures and it didn't work), node_modules, .pnp.cjs; as if it's a new machine for the project.

@emahuni you had to only remove the lockfile and node_modules. The reason you need to remove node_modules is because there is a duplicate of the lockfile inside node_modules/.pnpm/lock.yaml. The store should not be removed.

yes, that's right, but that didn't work. The reason why I had to remove it is a long story, but it worked at the end.

I was having that issue, tried what you are saying and more (deleting all lock files, .pnp, node_modules, and everything as if I am using pnpm for the first time - I actually had to write a script to do this since i did this many many times in trial and error mode trying to see where I was hitting a wall, coz it initially worked). So I ended up:

  • completely ditching pnpm (actually removing it and all its stores etc),
  • went to yarn 3, ended up developing a transitive peer dependency installer coz yarn doesn't have peer deps installation,
  • then was very annoyed by yarn 3 as it departs from yarn 1 usefulness, as you can't even do certain things anymore, it's a mess and probably the death of Yarn.
  • went to npm 8... grew a beard using it, so had to stop my accelerated ageing and...,
  • came back to pnpm thinking where did it go wrong coz one time it worked. Then it just worked again.
  • So I noticed that the issue must have been with the store, which one thing I wasn't touching.
  • later on... it happened again, so I just deleted everything by running that same said script and the store this time, and it worked... so yeah there is something happening there. It's as if pnpm thinks my workspace is a CI or something when it does, I am not even using any CI pipeline anywhere for that matter. This is the reason how I came to find this issue after I had had to deal with it more twice, I wanted to see if anyone was aware this was happening and throw my 2 cents at it.

However, this is now my package manager of choice coz it's just super super duper badass! Though I feel there are still a few edge cases to fix, the whole concept is going in the correct direction. cough support for something like yarn publish on v1 is needed pliz cough 😬

@emahuni
Copy link

emahuni commented Sep 12, 2022

yep, got this and had to delete pnpm entire store, my lock files (committed the deletion into vcs, I did this initially without these other measures and it didn't work), node_modules, .pnp.cjs; as if it's a new machine for the project.

@emahuni you had to only remove the lockfile and node_modules. The reason you need to remove node_modules is because there is a duplicate of the lockfile inside node_modules/.pnpm/lock.yaml. The store should not be removed.

yes, that's right, but it didn't work. The reason why I had to remove it is a long story, but it worked at the end.

I was having that issue, tried what you are saying and more (deleting all lock files, .pnp, node_modules, and everything as if I am using pnpm for the first time - I actually had to write a script to do this since i did this many many times in trial and error mode trying to see where I was hitting a wall, coz it initially worked). So I ended up:

  • completely ditching pnpm (actually removing it and all its stores etc),
  • went to yarn 3, ended up developing a transitive peer dependency installer coz yarn doesn't have peer deps installation,
  • then was very annoyed by yarn 3 as it departs from yarn 1 usefulness, as you can't even do certain things anymore, it's a mess and probably the death of Yarn.
  • went to npm 8... grew a beard using it, so had to stop my accelerated ageing and...,
  • came back to pnpm thinking where did it go wrong coz one time it worked. Then it just worked again.
  • So I noticed that the issue must have been with the store, which one thing I wasn't touching.
  • later on... it happened again, so I just deleted everything by running that same said script and the store this time, and it worked... so yeah there is something happening there. It's as if pnpm thinks my workspace is a CI or something when it does, I am not even using any CI pipeline anywhere for that matter. This is the reason how I came to find this issue after I had had to deal with it more twice, I wanted to see if anyone was aware this was happening and throw my 2 cents at it.

However, this is now my package manager of choice coz it's just super super duper badass! Though I feel there are still a few edge cases to fix, the whole concept is going in the correct direction. cough support for something like yarn publish on v1 is needed pliz cough 😬

Oh yah one thing I forgot to mention is that when I came back to pnpm, I had moved the .npmrc file into a bck dir when I ditched pnpm, so I was starting afresh.

you see that?

auto-install-peers=true
shamefully-hoist=true
node-linker=hoisted
strict-peer-dependencies=false

symlink=true
prefer-frozen-lockfile=false

shared-workspace-lockfile=false
prefer-workspace-packages=true
save-workspace-protocol=false
link-workspace-packages=deep

that was me trying to fix the issues before ditching pnpm, then this:

auto-install-peers=true
strict-peer-dependencies=false
link-workspace-packages=deep

is now, working without any issues when I came back to pnpm. believe me, I had that config before and it didn't work, it wasn't about this config.

@zkochan
Copy link
Member

zkochan commented Sep 15, 2022

I think the original issue was fixed. If you have exact steps to reproduce, open a new issue.

@zkochan zkochan closed this as completed Sep 15, 2022
@tleunen
Copy link
Author

tleunen commented Sep 22, 2022

@zkochan Is there any verbose mode or anything that could help finding the root cause? I'm having hard time reproducing it in a smaller testcase than what I have at work. But I might have found out that the lockfile is not generated/updated when I turn auto-install-peers on (auto-install-peers=true in .npmrc).
I confirmed that by deleting the file and running pnpm install again and nothing is generated. (tested with recent v7.12.1)

@emahuni
Copy link

emahuni commented Sep 22, 2022

@zkochan Is there any verbose mode or anything that could help finding the root cause? I'm having hard time reproducing it in a smaller testcase than what I have at work. But I might have found out that the lockfile is not generated/updated when I turn auto-install-peers on (auto-install-peers=true in .npmrc). I confirmed that by deleting the file and running pnpm install again and nothing is generated. (tested with recent v7.12.1)

Are you by any chance using workspaces at work?

@zkochan
Copy link
Member

zkochan commented Sep 22, 2022

Maybe you have this issue: #5372 (comment)

@callmeberzerker
Copy link

I am still hitting this issue in 7.14.1. Everything works locally. Can we see which package is outdated?

@HopeKenga
Copy link

I'm hitting with 7.17.0 and have followed all the processes but the workflow is still failing

@NullVoxPopuli
Copy link
Contributor

NullVoxPopuli commented Feb 14, 2023

I'm getting this error as well. hasBin: true is getting removed: https://github.com/CrowdStrike/ember-headless-form/actions/runs/4175506300/jobs/7230511766#step:4:192

which... makes sense.. most of these deps don't have bins.
But why is local pnpm, (also 7.27.0), adding hasBin: true?

This is how I tested, and made C.I. show me what is wrong:

        pnpm install --fix-lockfile
        git_diff=$(git diff) # this prints the git diff as well as stores it in a variable
        if [[ "$git_diff" -eq "" ]]; then
          echo "Success: no lockfile differences" ;
        else
          echo "Error: lockfile differences detected";
        fi

added here: https://github.com/CrowdStrike/ember-headless-form/pull/45/files#diff-423ef59ce8dec1c2647bc93466fa66fd1eb65a9a65a631d4730fdb753b0ef792

@WINOFFRG
Copy link

WINOFFRG commented May 7, 2023

Hi! Can anyone please help me with this issue
I am literally fed up of this, Builds and Installation all succeeds on local env but on github actions repeatedly getting the issue
https://github.com/WINOFFRG/limeplay/actions/runs/4909772277/jobs/8766378903

I have tried all the steps recommended by the community above. Waiting for response!

@zkochan
Copy link
Member

zkochan commented May 7, 2023

Try to use pnpm v8

https://github.com/WINOFFRG/limeplay/blob/77215e95b258b3d972e064f4bbcbf7fbf4773b08/.github/composite-actions/install/action.yml#L10

@jjangga0214
Copy link

jjangga0214 commented Jun 8, 2023

@zkochan This issue appears on v8 as well.. I tested with 8.5 and 8.6

@wallace-sf
Copy link

I had the same problem, but then I realized that there was an outdated pnpm-lock.yaml file in my repository. At some point, I had added it to .gitignore. When I tried to deploy on Vercel, I encountered this error. After uploading a new version of the lockfile, the error disappeared

@Usnul
Copy link

Usnul commented Nov 14, 2023

Here's what worked for me:

pnpm i --fix-lockfile --config.auto-install-peers=false

Run this locally, then commit your updated lockfile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.