-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate release workflow to cargo-dist
#9559
Conversation
b368887
to
4f38fec
Compare
CodSpeed Performance ReportMerging #9559 will not alter performanceComparing Summary
|
1592437
to
b2bf601
Compare
|
43f9981
to
537bcd1
Compare
## Summary This PR adds a release workflow powered by `cargo-dist`. It's similar to the version that's PR'd in Ruff (astral-sh/ruff#9559), with the exception that it doesn't include the Docker build or the "update dependents" step for pre-commit.
4aa8349
to
4ec0a93
Compare
4ec0a93
to
fe8c8c2
Compare
fe8c8c2
to
3cd6d03
Compare
I'm planning to ship this as part of Ruff v0.4.0 (i.e., in the next few weeks) given that it changes the archive format. |
8b2fe3c
to
f54791f
Compare
I will merge this today. |
f54791f
to
eeefac1
Compare
eeefac1
to
c2d09d8
Compare
c2d09d8
to
ff2c9e8
Compare
Inherit profile Update target list Slim down release for testing Set repo Wrap in dummy dir Prep for PR Add Docker and dependents jobs Move to post-announce Add more docs Temp: testing Split up docker Update names Update cargo dist Use OCI Use a single docker task Uncomment
ff2c9e8
to
4557a13
Compare
## Summary This PR migrates our release workflow to [`cargo-dist`](https://github.com/axodotdev/cargo-dist). The primary motivation here is that we want to ship dedicated installers for Ruff that work across platforms, and `cargo-dist` gives us those installers out-of-the-box. The secondary motivation is that `cargo-dist` formalizes some of the patterns that we've built up over time in our own release process. At a high level: - The `release.yml` file is generated by `cargo-dist` with `cargo dist generate`. It doesn't contain any modifications vis-a-vis the generated file. (If it's edited out of band from generation, the release fails.) - Our customizations are inserted as custom steps within the `cargo-dist` workflow. Specifically, `build-binaries` builds the wheels and packages them into binaries (as on `main`), while `build-docker.yml` builds the Docker image. `publish-pypi.yml` publishes the wheels to PyPI. This is effectively our `release.yaml` (on `main`), broken down into individual workflows rather than steps within a single workflow. ### Changes from `main` The workflow is _nearly_ unchanged. We kick off a release manually via the GitHub Action by providing a tag. If the tag doesn't match the `Cargo.toml`, the release fails. If the tag matches an already-existing release, the release fails. The release proceeds by (in order): 0. Doing some upfront validation via `cargo-dist`. 1. Creating the wheels and archives. 2. Building and pushing the Docker image. 3. Publishing to PyPI (if it's not a "dry run"). 4. Creating the GitHub Release (if it's not a "dry run"). 5. Notifying `ruff-pre-commit` (if it's not a "dry run"). There are a few changes in the workflow as compared to `main`: - **We no longer validate the SHA** (just the tag). It's not an input to the job. The Axo team is considering whether / how to support this. - **Releases are now published directly** (rather than as draft). Again, the Axo team is considering whether / how to support this. The downside of drafts is that the URLs aren't stable, so the installers don't work _as long as the release is in draft_. This is fine for our workflow. It seems like the Axo team will add it. - Releases already contain the latest entry from the changelog (we don't need to copy it over). This "Just Works", which is nice, though we'll still want to edit them to add contributors. There are also a few **breaking changes** for consumers of the binaries: - **We no longer include the version tag in the file name**. This enables users to install via `/latest` URLs on GitHub, and is part of the cargo-dist paradigm. - **Archives now include an extra level of nesting,** which you can remove with `--strip-components=1` when untarring. Here's an example release that I created -- I omitted all the artifacts since I was just testing a workflow, so none of the installers or links work, but it gives you a sense for what the release looks like: https://github.com/charliermarsh/cargodisttest/releases/tag/0.1.13. ### Test Plan I ran a successful release to completion last night, and installed Ruff via the installer: ![Screenshot 2024-01-17 at 12 12 53 AM](https://github.com/astral-sh/ruff/assets/1309177/a5334466-2ca3-4279-a453-e912a0805df2) ![Screenshot 2024-01-17 at 12 12 48 AM](https://github.com/astral-sh/ruff/assets/1309177/63ac969e-69a1-488c-8367-4cb783526ca7) The piece I'm least confident about is the Docker push. We build the image, but the push fails in my test repo since I haven't wired up the credentials.
## Summary This PR migrates our release workflow to [`cargo-dist`](https://github.com/axodotdev/cargo-dist). The primary motivation here is that we want to ship dedicated installers for Ruff that work across platforms, and `cargo-dist` gives us those installers out-of-the-box. The secondary motivation is that `cargo-dist` formalizes some of the patterns that we've built up over time in our own release process. At a high level: - The `release.yml` file is generated by `cargo-dist` with `cargo dist generate`. It doesn't contain any modifications vis-a-vis the generated file. (If it's edited out of band from generation, the release fails.) - Our customizations are inserted as custom steps within the `cargo-dist` workflow. Specifically, `build-binaries` builds the wheels and packages them into binaries (as on `main`), while `build-docker.yml` builds the Docker image. `publish-pypi.yml` publishes the wheels to PyPI. This is effectively our `release.yaml` (on `main`), broken down into individual workflows rather than steps within a single workflow. ### Changes from `main` The workflow is _nearly_ unchanged. We kick off a release manually via the GitHub Action by providing a tag. If the tag doesn't match the `Cargo.toml`, the release fails. If the tag matches an already-existing release, the release fails. The release proceeds by (in order): 0. Doing some upfront validation via `cargo-dist`. 1. Creating the wheels and archives. 2. Building and pushing the Docker image. 3. Publishing to PyPI (if it's not a "dry run"). 4. Creating the GitHub Release (if it's not a "dry run"). 5. Notifying `ruff-pre-commit` (if it's not a "dry run"). There are a few changes in the workflow as compared to `main`: - **We no longer validate the SHA** (just the tag). It's not an input to the job. The Axo team is considering whether / how to support this. - **Releases are now published directly** (rather than as draft). Again, the Axo team is considering whether / how to support this. The downside of drafts is that the URLs aren't stable, so the installers don't work _as long as the release is in draft_. This is fine for our workflow. It seems like the Axo team will add it. - Releases already contain the latest entry from the changelog (we don't need to copy it over). This "Just Works", which is nice, though we'll still want to edit them to add contributors. There are also a few **breaking changes** for consumers of the binaries: - **We no longer include the version tag in the file name**. This enables users to install via `/latest` URLs on GitHub, and is part of the cargo-dist paradigm. - **Archives now include an extra level of nesting,** which you can remove with `--strip-components=1` when untarring. Here's an example release that I created -- I omitted all the artifacts since I was just testing a workflow, so none of the installers or links work, but it gives you a sense for what the release looks like: https://github.com/charliermarsh/cargodisttest/releases/tag/0.1.13. ### Test Plan I ran a successful release to completion last night, and installed Ruff via the installer: ![Screenshot 2024-01-17 at 12 12 53 AM](https://github.com/astral-sh/ruff/assets/1309177/a5334466-2ca3-4279-a453-e912a0805df2) ![Screenshot 2024-01-17 at 12 12 48 AM](https://github.com/astral-sh/ruff/assets/1309177/63ac969e-69a1-488c-8367-4cb783526ca7) The piece I'm least confident about is the Docker push. We build the image, but the push fails in my test repo since I haven't wired up the credentials.
Summary
This PR migrates our release workflow to
cargo-dist
. The primary motivation here is that we want to ship dedicated installers for Ruff that work across platforms, andcargo-dist
gives us those installers out-of-the-box. The secondary motivation is thatcargo-dist
formalizes some of the patterns that we've built up over time in our own release process.At a high level:
release.yml
file is generated bycargo-dist
withcargo dist generate
. It doesn't contain any modifications vis-a-vis the generated file. (If it's edited out of band from generation, the release fails.)cargo-dist
workflow. Specifically,build-binaries
builds the wheels and packages them into binaries (as onmain
), whilebuild-docker.yml
builds the Docker image.publish-pypi.yml
publishes the wheels to PyPI. This is effectively ourrelease.yaml
(onmain
), broken down into individual workflows rather than steps within a single workflow.Changes from
main
The workflow is nearly unchanged. We kick off a release manually via the GitHub Action by providing a tag. If the tag doesn't match the
Cargo.toml
, the release fails. If the tag matches an already-existing release, the release fails.The release proceeds by (in order):
cargo-dist
.ruff-pre-commit
(if it's not a "dry run").There are a few changes in the workflow as compared to
main
:There are also a few breaking changes for consumers of the binaries:
/latest
URLs on GitHub, and is part of the cargo-dist paradigm.--strip-components=1
when untarring.Here's an example release that I created -- I omitted all the artifacts since I was just testing a workflow, so none of the installers or links work, but it gives you a sense for what the release looks like: https://github.com/charliermarsh/cargodisttest/releases/tag/0.1.13.
Test Plan
I ran a successful release to completion last night, and installed Ruff via the installer:
The piece I'm least confident about is the Docker push. We build the image, but the push fails in my test repo since I haven't wired up the credentials.