This crate hasn't been released in almost 3 months

[fasterthanlime]

#3287 was merged on July 16, which is great.

We needed it to implement additional functionality in the nix crate. But the nix crate obviously cannot have a "git dependency" on libc, it needs to point at something released on crates.io.

(This is all in service of https://lib.rs/crates/ktls which uses some APIs I guess nobody else thought to include in the libc crate because they're a bit niche),

I have no idea who maintains the libc crate, if there's a release schedule, who to talk to about this.

I understand reviewing PRs takes time but.. surely releasing whatever's in the default branch is somewhat automated? Or at least quick enough?

Let me know how to proceed, I'm at a complete loss here.

[yerke]

cc @JohnTitor

[joshtriplett]

@fasterthanlime Unfortunately, the release process is in fact not automated at all, and requires someone with publish permissions on the crate (anyone in the crate-maintainers team) to manually bump the version in Cargo.toml, send that in as a PR (or merge someone else's PR doing so), make and push a tag, and cargo publish.

A subset of the folks on crate-maintainers do specifically review and merge PRs on libc, but there's a process gap here in that it isn't specifically anyone's repsonsibility in particular to consider making a release, nor is there any defined regular cadence.

There are some plans for better automation in the works, now that https://blog.rust-lang.org/2023/06/23/improved-api-tokens-for-crates-io.html is available.

Meanwhile, I'll get a release out.

[fasterthanlime]

Happy to do that now.

I'd greatly appreciate that, thanks!

the release process is in fact not automated

Would there be interest in automating it? Is that something I can help with? I feel like this would be a good use of the rest of my 2023 Rust foundation grant (if not no big deal, plenty of other projects need help).

[joshtriplett]

@fasterthanlime I'd suggest coordinating with infra on that to check in (as I think others might also be looking at it and may already have some work in that direction). I've certainly heard various comments to the effect that many people (for many different crates) would be interested in some form of automated make-a-publish-happen workflow (which would need to be determined/proposed).

[fasterthanlime]

I'd suggest coordinating with infra

Is the best way to reach "infra" the #t-infra stream on zulip?

[joshtriplett]

I'd expect that to be the best starting point to discuss automating crate publishes, yeah.

[fasterthanlime]

@joshtriplett thanks.

Re the 0.2.148 release: I believe it's missing a Git tag / GitHub release with a changelog, see previous releases here: https://github.com/rust-lang/libc/releases

[joshtriplett]

@fasterthanlime There's a signed tag, just not a release with release notes.

cc @JohnTitor or others for adding the GitHub release (also not automated).

[mgeisler]

Hey folks,

Re the automated releases, there are many ways to do this — I've been using https://github.com/mgeisler/rust-release-gh-action for my own libraries. It works like this:

1. I push a new branch called release-x.y.z to GitHub
2. The prepare-release GH action sees this and if x.y.z is different from the version number in Cargo.toml, it does a few things:
   1. Updates the changelog. The changelog is a list with the titles of all pull requests that have been merged since the last release. All features and bug fixes go through pull requests in my repositories so I find this a useful logical unit.
   2. I added optional machinery to generate a dependency graph with cargo-depgraph, but that's probably not for everybody.
   3. Updates the version number to x.y.z in Cargo.toml and in the README.md (if found).
   4. Pushes these commits to the release-x.y.z branch.
   5. Opens a PR back to your main branch.
3. I pull the new commits on the release-x.y.z branch and I then edit the changelog to my liking. I remove unimportant PRs and I rephrase things as needed. Nothing has been tagged, nothing has been published, so I have complete freedom to fixup things here.
4. After tests have passed on the PR, I merge it.
5. The publish-crate GH action sees that the version number in Cargo.toml doesn't have a GitHub tag. It then does the following
   1. Runs cargo publish
   2. Extracts the changelog commited earlier and uses it to create a GitHub releases. This also adds a Git tag.

What I like about this setup is that it automates the tedious bits, but leaves room for me to adjust a release as needed. Because the state of the release is in its own branch, I can fix typos in READMEs and changelogs as needed and have the whole thing tested with the normal CI setup. I like that the Git tag is added on the code after running cargo publish

I know there are other tools for automating this. I feel that cargo-release is lacking the two-step process. Looking at the related tools on their page, I see release-pr Action, which looks close in spirit to my setup.

Maybe the ideas here could be useful to someone — they're fairly easy to reimplement or extend to your needs.

[Manishearth]

Yeah, I'd also say that the initiating step for releasing libraries should not be automated since there are a bunch of manual judgements that go into making the call whether something is ready to release (or needs release at all) and if this crate ever hits 1.0 (which I hope it tries for), minor vs patch versioning.

The toolchain has effectively clockwork scheduled releases but a lot of work is put into making that feasible; I don't think other libraries have put in sufficient work to make that a thing. Not just libc, I'm talking about the bulk of the ecosystem.

That said, I'm only talking about the initiating step, the process described by @mgeisler still has manual initiation and automated everything-else which seems great. (certainly something I may look into for my own projects since I hate doing the release two-step)

---

Definitely also having better documentation on who is in charge of a release and how to get their attention is also going to help. There are many places that one might say something to attempt to get a rust team's attention and it's never clear which is the one that team actually pays attention to.

[fasterthanlime]

Re the automated releases, there are many ways to do this — I've been using https://github.com/mgeisler/rust-release-gh-action for my own libraries. It works like this:

This sounds like an interesting experiment to run against one of the crates maintained by "the rust libs team". Maybe a smaller one than libc at first? Or maybe set everything up but it publishes to "libc-automated-releases-tryouts" on crates.io or something?

The sticking points here are, I'd imagine:

• The libs team doesn't want to give just anyone permission to publish the libc crate on crates.io (for good reason)
• The libs team doesn't want a "bad" release of libc to end up on crates.io (although, I guess another manual release could be done shortly thereafter + the bad release could be yanked, it's not MUCH harm done as long as we're reactive)

One cautious course of action could be to set up a dummy crate repo somewhere else, so folks currently involved with releasing "official rust libs crates" can play around with the workflow @mgeisler suggested (and/or other similar workflows), making them confident it'll work for bigger/real crates.

How does that sound?

[mgeisler]

Re the automated releases, there are many ways to do this — I've been using https://github.com/mgeisler/rust-release-gh-action for my own libraries. It works like this:

This sounds like an interesting experiment to run against one of the crates maintained by "the rust libs team". Maybe a smaller one than libc at first? Or maybe set everything up but it publishes to "libc-automated-releases-tryouts" on crates.io or something?

One thing my little contraption lacks is tests... partly because I don't know how to test publish a package. Now that I search for it, I found https://staging.crates.io/ — is that a place where one can test-publish crates?

Note that the main point of my comment was to float the idea used here (separation between preparing a release and actually publishing it, always using reversible actions until the final cargo publish). The implementation is fairly simply, just ~250 lines of JavaScript and Python inside the YAML files. I would like to write it in Rust instead for my own sanity and for ease of development 😄

[fasterthanlime]

Just following up on this, although it'd probably be best to move that conversation somewhere else in the near future.

I've tried a couple "crate release automation" solutions since we last spoke, and settled on https://release-plz.ieni.dev by @MarcoIeni.

Here's an example release PR: bearcove/ktls#35 (automatically opened + updated, step 1 in the two-step release process).

You can see that I used to maintain the changelog manually and that release-plz took over: https://github.com/hapsoc/ktls/blame/main/CHANGELOG.md - of course I could've made the effort to edit it before merging the "release" PR, which I'm assuming is what will happen on "rust libs" crates.

The releases themselves also contain changelog items: https://github.com/hapsoc/ktls/releases

And they're automatically published to crates.io: https://crates.io/crates/ktls/versions

release-plz also uses cargo-semver (and conventional commits, if you use them) to determine if the version bump should be major/minor/patch.

As far as I can tell, release-plz (an actual Rust executable running in CI, comes with its own GitHub Action) is best-of-class. The changelog management part might be the sticking point here, also "proving it's fine to run this binary with a token that's able to publish an important crate such as libc", but I've briefly spoken with @MarcoIeni and they are interested in supporting the needs of the libs team.

I'm happy to act as a facilitator (or not, if none is needed!) to have that happen. It probably involves heading over to Zulip (my nemesis!) but before I do, can I get a quick vibe check from @JohnTitor and @joshtriplett (+ feel free to tag whoever else should be looking at this)

[mgeisler]

I've tried a couple "crate release automation" solutions since we last spoke, and settled on https://release-plz.ieni.dev by @MarcoIeni.

I actually saw this recently after you tweeted about it — it looks really nice and from skimming the docs, I think it has a similar philosophy to what I've been using. So I'm planning on trying it for my own crates too 😄