Cargo: proposed new rangeStrategy

[sunshowers]

What would you like help with?

Other

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

GitHub, 37.280.0

Please tell us more about your question or problem

Hi -- we have a Renovate setup going over at Oxide for our Rust and npm dependencies, and we really love it!

I have:

1. Some questions about how to achieve a goal.
2. If my understanding of the current world is correct, a proposal for 1-2 new range strategies to be added to Renovate.
3. In addition, a proposal to also change the default from "bump" to one of these new strategies.

Context

Oxide is a Rust-heavy company, though we also use npm/JavaScript in a few places. I'm going to focus on Rust in this discussion because that's what I'm most familiar with.

We sometimes specify Rust dependencies in this fashion:

[dependencies]
chrono = "0.4"
syn = "2"

And we almost always check in Cargo.lock as well.

(Note that for Cargo, "0.4" actually means "~0.4", and "2" or "2.0" means "^2.0.0".)

A thing we've observed is that by default, we don't get Renovate PRs for updates if the full version (x.y.z) isn't specified. So, for example, we currently have chrono 0.4.34 in one of our repos, but haven't gotten a PR to update to version 0.4.37.

I did some testing and it looks like there's an "update-lockfile" strategy that does achieve this goal. But that can't be globally specified because if a full version x.y.z is specified in Cargo.toml, then that doesn't get bumped with this strategy.

This is easiest to summarize in a table. Below, we assume that version x.y.z is being updated to u.v.w, where the versions may be compatible or incompatible (e.g. u may be the same as x, or not.)

rangeStrategy	spec	update type	update Cargo.lock	update Cargo.toml
bump	x or x.y	compatible	no	no
bump	x or x.y	incompatible	yes	yes, to u or u.v respectively
bump	x.y.z	compatible	yes	yes, to u.v.w
bump	x.y.z	incompatible	yes	yes, to u.v.w
update-lockfile	x or x.y	compatible	yes	no
update-lockfile	x or x.y	incompatible	yes	yes, to u or u.v respectively
update-lockfile	x.y.z	compatible	yes	no
update-lockfile	x.y.z	incompatible	yes	yes, to u.v.w

As a workaround, we've come up with a hybrid strategy. This strategy makes it so that if versions are specified as x or x.y, then we use the update-lockfile strategy. Otherwise, we use the bump strategy. We've implemented this via a regex, see this config.

For the hybrid strategy, we have:

rangeStrategy	spec	update type	update Cargo.lock	update Cargo.toml
hybrid	x or x.y	compatible	yes	no
hybrid	x or x.y	incompatible	yes	yes, to u or u.v respectively
hybrid	x.y.z	compatible	yes	yes, to u.v.w
hybrid	x.y.z	incompatible	yes	yes, to u.v.w

And this strategy works well enough. But ideally, we actually want to always change the version in Cargo.toml. In other words, we would like to have this strategy:

rangeStrategy	spec	update type	update Cargo.lock	update Cargo.toml
desired	x or x.y	compatible	yes	yes, to u.v.w
desired	x or x.y	incompatible	yes	yes, to u.v.w
desired	x.y.z	compatible	yes	yes, to u.v.w
desired	x.y.z	incompatible	yes	yes, to u.v.w

To the best of my knowledge, this desired strategy does not exist -- I couldn't find a set of flags which would enforce the strategy.

Questions

Based on the above discussions, I have a few questions:

1. Is there an easier way to achieve the "hybrid" strategy mentioned above? I'm glad it is possible to configure this, but based on my reading of the Renovate source code there didn't seem to be a way to achieve that in a simpler fashion.
2. Does the "desired" strategy above make sense? I couldn't see how to achieve this at all based on my read of the documentation and Renovate source code. (It's quite possible I've missed something, though!)
3. If the answer to 1 or 2 is "no", would you consider adding these as additional rangeStrategy options?
4. I really feel like the hybrid strategy above should be the default at least for Cargo. It's pretty surprising that some lockfile updates just get ignored. What do you think?

I hope the above description is detailed enough. Always happy to provide more details!

Other notes

A repo where I ran some tests is at https://github.com/sunshowers/renovate-test. This test repo uses the hybrid strategy, and you can see the results in the list of PRs.

The periodic lockFileMaintenance job sadly hasn't been working for us because it's all-or-nothing: there are some newer versions of deps that are compatible and some that aren't (particularly git deps), so updating them one-by-one is necessary.

Again, I want to say that Renovate is incredibly cool and we overall love our setup. I hope this request can help improve things for all Renovate users.

Logs (if relevant)

Logs

Here's an example log for the repo linked above:

https://github.com/sunshowers/renovate-test/blob/main/example.log

I ran this with

docker run -v ./config.json:/config.json -e RENOVATE_TOKEN=***** -e RENOVATE_CONFIG_FILE=/config.json -e LOG_LEVEL=debug -e RENOVATE_PR_HOURLY_LIMIT=10 --rm renovate/renovate

[rarkins]

Thanks for your thoughtful analysis. Trying to break this into some smaller pieces, we can go back to other parts later if necessary.

If rangeStrategy=update-lockfile and there's an in-range update (e.g. from 1.2.3 to 1.2.4) then you should get a PR. Are you sure it's not because you had 1.2.4 in your lockfile, i.e. you're already updated? If you have 1.2.3 in Cargo.toml/lock and no update for a 1.2.4, please create a reproduction repo so we can treat that as a bug.

Next, I think that rangeStrategy=auto should default to update-lockfile instead of bump for the cargo manager. i.e. update the locked version by default but not keep bumping the version in Cargo.toml

[rarkins]

Addressed in #28632

[sunshowers]

Thank you! Let me try that now.

[sunshowers]

This is awesome! It looks like bump now creates PRs in all cases. To go back and update the table:

rangeStrategy	spec	update type	update Cargo.lock	update Cargo.toml
new bump	x or x.y	compatible	yes (old: no)	yes, to u.v.w (old: no) (example 1)
new bump	x or x.y	incompatible	yes	yes, to u.v.w (old: to u or u.v) (example 2)
new bump	x.y.z	compatible	yes	yes, to u.v.w
new bump	x.y.z	incompatible	yes	yes, to u.v.w

This matches our desired strategy exactly. Thanks!

There's still this outstanding point:

Next, I think that rangeStrategy=auto should default to update-lockfile instead of bump for the cargo manager. i.e. update the locked version by default but not keep bumping the version in Cargo.toml

Is this something you'd still like to do? I personally prefer bump but would understand if the default was update-lockfile.

[rarkins]

Yes, I still plan to do it. Essentially it means that by default, we assume that if your Cargo.toml has a range (e.g. 1.0.0) and not a pinned version (=1.0.0) then we assume you want to keep that range and not have it bumped/narrowed. It's a bit too opinionated to bump by default because it's not suitable for most libraries.

[sunshowers]

All right, happy to close the discussion since our specific problem has been addressed, or leave it open if you'd like. Thanks so much again!

[kornelski]

This change has caused unwanted churn for me ;(

Cargo.toml specifies minimum allowed versions, rather than the exact versions of a package to use, so it doesn't need to be updated as often as Cargo.lock.

Updating Cargo.toml is more disruptive than just updating Cargo.lock, because Cargo.toml can be manually edited to add dependencies or toggle optional features, so needless version bumps can cause merge conflicts. Because Cargo.toml is a source of truth, these merge conflicts must be resolved manually. OTOH Cargo.lock can be easily rebuilt, so it's not a big deal when it conflicts.

I suggest at least don't bump semver-patch versions in Cargo.toml for versions above 1.0.0, unless in response to security patches.

[sunshowers]

@kornelski could you use the update-lockfile strategy?

(edit: fixed word order :) )

[rarkins]

Yes, please switch from the default rangeStrategy=auto to either rangeStrategy=replace (which would mean no updates at all unless package file needs changing) or rangeStrategy=update-lockfile (which means updates only to the lock file, unless package file also need updating such as for major updates)