Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support rangeStrategy=update-lockfile for pnpm #21438

Closed
rarkins opened this issue Apr 12, 2023 · 21 comments · Fixed by #26770
Closed

Support rangeStrategy=update-lockfile for pnpm #21438

rarkins opened this issue Apr 12, 2023 · 21 comments · Fixed by #26770
Assignees
Labels
manager:npm package.json files (npm/yarn/pnpm) priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)

Comments

@rarkins
Copy link
Collaborator

rarkins commented Apr 12, 2023

What would you like Renovate to be able to do?

As per above

If you have any ideas on how this should be implemented, please tell us here.

Depends on #19038

Then should add support for updateLockedDependency()

Is this a feature you are interested in implementing yourself?

No

@rarkins rarkins added type:feature Feature (new functionality) priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others manager:npm package.json files (npm/yarn/pnpm) status:blocked Issue is blocked by another issue or external requirement labels Apr 12, 2023
@DavidS-ovm
Copy link

#19038 is now closed fixed, so this might be unblocked now.

As a pnpm shop, this is quite the blocker for us.

@rarkins
Copy link
Collaborator Author

rarkins commented Jul 21, 2023

PRs welcome

@rarkins rarkins added status:ready and removed status:blocked Issue is blocked by another issue or external requirement labels Aug 14, 2023
@pataar
Copy link
Contributor

pataar commented Nov 16, 2023

@rarkins I'd like to give it a try. Can you give a nudge in the right direction? Where can I find the yarn.lock parsing for example?

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 16, 2023

Hi @pataar! Thanks for offering.

Perhaps the most important part doesn't require knowledge of Renovate code but rather of how pnpm works.

The Scenario is:

  • There's a range constraint for a dependency in package.json, e.g. ^1.0.0
  • The pnpm lock file resolves this to a version in-range, such as 1.2.3
  • The latest version is 1.5.0, but we wish to upgrade it to 1.3.0 specifically

In this case there's usually three ways to approach it.

  1. Occasionally a package manager is featureful enough that you can ask it to do this via a command-line argument, e.g. like pnpm upgrade foo 1.3.0.
  2. Instead we can lightly edit the lock file and then run e.g. pnpm lock and have it update the remaining parts of the lock file (e.g. in case transitive dependencies changed).
  3. We can potentially manipulate the package.json to reference 1.3.0 instead of ^1.0.0, run pnpm to update the lock file, then revert the package.json change and run it again, and.. it works

So you need to research:

  • Is there any pnpm command which can let us "surgically" update a locked version? (and btw the pnpm maintainer is wonderful and might be open to adding this feature if not)
  • Or, does pnpm let us mutate the lock file manually and then it "repairs" it for us (e.g. fills in sha's, changes transitive dependencies if necessary, etc)
  • Or, can we do the package.json manipulation trick

@alan910127
Copy link

Hello, a random guy who hopes this feature can be realized here.

Based on my (limited) knowledge, we can either use add, install, or update with the syntax pkg@version (foo@1.3.0 for example) to upgrade a dependency to a specified version.
However, all of the above modifies the package.json, but it seems to work with the third approach mentioned above.

@pataar
Copy link
Contributor

pataar commented Nov 17, 2023

Hello, a random guy who hopes this feature can be realized here.

Based on my (limited) knowledge, we can either use add, install, or update with the syntax pkg@version (foo@1.3.0 for example) to upgrade a dependency to a specified version. However, all of the above modifies the package.json, but it seems to work with the third approach mentioned above.

What if we save the state of package.json before executing those commands, and restoring it afterwards. And simply run pnpm install to make the right corrections to the lockfile? Worth trying at least.

// edit: that's basically what @rarkins said in his approach examples

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 17, 2023

Yes:

  • Manipulate the package.json
  • Run pnpm install
  • Revert the package.json manipulation
  • Run pnpm install again
  • Verify it's still like we want

@alan910127
Copy link

alan910127 commented Nov 17, 2023

When I left that comment a few hours ago, I literally went through the install-revert-install process to update one of my dependencies to a newer, but not latest, version. And it works just as we expected.

IMO, even though it is not a graceful solution, it is totally feasible.

@segevfiner
Copy link

I think pnpm update does essentially something similar except the revert the package.json part, not sure if it has a --no-save flag to do the change without modifying the package.json, maybe it does?

@alan910127
Copy link

@segevfiner I just tried it, and it does that! 😄

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 17, 2023

Please check that pnpm update allows exact versions to be specified, otherwise it's not what we need

@alan910127
Copy link

I think this shows that it is what we need:

image

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 17, 2023

That looks like the type of update which would change package.json, unless you had it set to a range like *. Better to pick a non-major update example to test with

@alan910127
Copy link

alan910127 commented Nov 17, 2023

This should be a better example:

image

Edit:

package.json

@rarkins rarkins added the auto:reproduction A minimal reproduction is necessary to proceed label Nov 17, 2023
Copy link
Contributor

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the discussion if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 17, 2023

Someone please create a reproduction repo demonstrating a repo where:

  • package.json uses a range
  • there are at least 2 versions out of date

@alan910127
Copy link

Just opened a reproduction repo: renovate-reproduction-21438.

Please let me know if there is something I missed.

@rarkins
Copy link
Collaborator Author

rarkins commented Nov 22, 2023

Reproduction forked to https://github.com/renovate-reproductions/21438

@rarkins rarkins removed the auto:reproduction A minimal reproduction is necessary to proceed label Nov 22, 2023
@uberflip-jbooth
Copy link

I've done a bit of testing on my own repo and pnpm update "<package_name>@<version>" --no-save makes a lot more changes to pnpm-lock.yaml than pnpm add "<package_name>@<version>". The latter seems more like the expected behaviour, the only catch is reverting package.json afterwards.

@guillaumeduboc
Copy link
Contributor

I've been looking into this these past days
First of all thanks for the repro example and the --no-save for pnpm !

I have a working pr #26770 but I am not completely sure about the implementation inside renovate. Especially I don't get the difference between

kudoas added a commit to kudoas/portfolio that referenced this issue Feb 9, 2024
kudoas added a commit to kudoas/html-table-creator that referenced this issue Feb 9, 2024
@renovate-release
Copy link
Collaborator

🎉 This issue has been resolved in version 37.207.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
manager:npm package.json files (npm/yarn/pnpm) priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants