Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate release workflow to cargo-dist #9559

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Jan 17, 2024

Summary

This PR migrates our release workflow to cargo-dist. The primary motivation here is that we want to ship dedicated installers for Ruff that work across platforms, and cargo-dist gives us those installers out-of-the-box. The secondary motivation is that cargo-dist formalizes some of the patterns that we've built up over time in our own release process.

At a high level:

  • The release.yml file is generated by cargo-dist with cargo dist generate. It doesn't contain any modifications vis-a-vis the generated file. (If it's edited out of band from generation, the release fails.)
  • Our customizations are inserted as custom steps within the cargo-dist workflow. Specifically, build-binaries builds the wheels and packages them into binaries (as on main), while build-docker.yml builds the Docker image. publish-pypi.yml publishes the wheels to PyPI. This is effectively our release.yaml (on main), broken down into individual workflows rather than steps within a single workflow.

Changes from main

The workflow is nearly unchanged. We kick off a release manually via the GitHub Action by providing a tag. If the tag doesn't match the Cargo.toml, the release fails. If the tag matches an already-existing release, the release fails.

The release proceeds by (in order):

  1. Doing some upfront validation via cargo-dist.
  2. Creating the wheels and archives.
  3. Building and pushing the Docker image.
  4. Publishing to PyPI (if it's not a "dry run").
  5. Creating the GitHub Release (if it's not a "dry run").
  6. Notifying ruff-pre-commit (if it's not a "dry run").

There are a few changes in the workflow as compared to main:

  • We no longer validate the SHA (just the tag). It's not an input to the job. The Axo team is considering whether / how to support this.
  • Releases are now published directly (rather than as draft). Again, the Axo team is considering whether / how to support this. The downside of drafts is that the URLs aren't stable, so the installers don't work as long as the release is in draft. This is fine for our workflow. It seems like the Axo team will add it.
  • Releases already contain the latest entry from the changelog (we don't need to copy it over). This "Just Works", which is nice, though we'll still want to edit them to add contributors.

There are also a few breaking changes for consumers of the binaries:

  • We no longer include the version tag in the file name. This enables users to install via /latest URLs on GitHub, and is part of the cargo-dist paradigm.
  • Archives now include an extra level of nesting, which you can remove with --strip-components=1 when untarring.

Here's an example release that I created -- I omitted all the artifacts since I was just testing a workflow, so none of the installers or links work, but it gives you a sense for what the release looks like: https://github.com/charliermarsh/cargodisttest/releases/tag/0.1.13.

Test Plan

I ran a successful release to completion last night, and installed Ruff via the installer:

Screenshot 2024-01-17 at 12 12 53 AM

Screenshot 2024-01-17 at 12 12 48 AM

The piece I'm least confident about is the Docker push. We build the image, but the push fails in my test repo since I haven't wired up the credentials.

@charliermarsh charliermarsh force-pushed the charlie/cargo-dist-test branch 2 times, most recently from b368887 to 4f38fec Compare January 17, 2024 05:47
@charliermarsh charliermarsh added the release Related to the release process label Jan 17, 2024
Copy link

codspeed-hq bot commented Jan 17, 2024

CodSpeed Performance Report

Merging #9559 will not alter performance

Comparing charlie/cargo-dist-test (f54791f) with main (6050bab)

Summary

✅ 30 untouched benchmarks

@charliermarsh charliermarsh force-pushed the charlie/cargo-dist-test branch 3 times, most recently from 1592437 to b2bf601 Compare January 17, 2024 06:09
Copy link
Contributor

github-actions bot commented Jan 17, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@charliermarsh charliermarsh marked this pull request as ready for review January 17, 2024 22:09
Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
.github/workflows/build-binaries.yml Outdated Show resolved Hide resolved
.github/workflows/build-binaries.yml Show resolved Hide resolved
.github/workflows/build-docker.yml Show resolved Hide resolved
charliermarsh added a commit to astral-sh/uv that referenced this pull request Jan 18, 2024
## Summary

This PR adds a release workflow powered by `cargo-dist`. It's similar to
the version that's PR'd in Ruff
(astral-sh/ruff#9559), with the exception that
it doesn't include the Docker build or the "update dependents" step for
pre-commit.
@charliermarsh charliermarsh force-pushed the charlie/cargo-dist-test branch 2 times, most recently from 4aa8349 to 4ec0a93 Compare February 8, 2024 14:52
@charliermarsh charliermarsh added the breaking Breaking API change label Feb 8, 2024
Inherit profile

Update target list

Slim down release for testing

Set repo

Wrap in dummy dir

Prep for PR

Add Docker and dependents jobs

Move to post-announce

Add more docs

Temp: testing

Split up docker

Update names

Update cargo dist

Use OCI

Use a single docker task

Uncomment
@charliermarsh
Copy link
Member Author

I'm planning to ship this as part of Ruff v0.4.0 (i.e., in the next few weeks) given that it changes the archive format.

@charliermarsh charliermarsh added this to the v0.4.0 milestone Apr 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking Breaking API change release Related to the release process
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants