-
Notifications
You must be signed in to change notification settings - Fork 990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: build relases on gh-actions & support linux arm64 #2629
Changes from 4 commits
6422779
2904f63
5bcf7c6
0ec2840
ea4b901
d411bd9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
name: Build & Test | ||
|
||
on: | ||
push: | ||
branches: ["*"] | ||
|
||
env: | ||
# Cross-compilation for aarch64 requires a different linker | ||
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc | ||
|
||
permissions: | ||
contents: read | ||
|
||
jobs: | ||
Tests: | ||
runs-on: ${{ matrix.os }} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
target: | ||
- x86_64-unknown-linux-gnu | ||
- aarch64-unknown-linux-gnu | ||
- x86_64-pc-windows-msvc | ||
- x86_64-apple-darwin | ||
- aarch64-apple-darwin | ||
rustup_toolchain: [stable] | ||
include: | ||
- os: windows-2022 | ||
target: x86_64-pc-windows-msvc | ||
- os: ubuntu-20.04 | ||
target: x86_64-unknown-linux-gnu | ||
- os: ubuntu-20.04 | ||
target: aarch64-unknown-linux-gnu | ||
- os: macos-13 | ||
target: x86_64-apple-darwin | ||
- os: macos-14 | ||
target: aarch64-apple-darwin | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install Rust (UNIX) | ||
run: | | ||
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain ${{ matrix.rustup_toolchain }} | ||
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin" | ||
# We need to crosscompile for arm64 linux for now | ||
if [[ "${{ matrix.target }}" == "aarch64-unknown-linux-gnu" ]]; then | ||
echo "Adding aarch64-unknown-linux-gnu target" | ||
sudo apt-get update -y | ||
sudo apt-get install -y make g++ libssl-dev gcc-aarch64-linux-gnu | ||
rustup target add aarch64-unknown-linux-gnu | ||
else | ||
echo "Compiling for ${{matrix.target}}" | ||
fi | ||
if: ${{ ! startsWith(matrix.os, 'windows') }} | ||
|
||
- name: Install Rust (Windows) | ||
run: | | ||
curl -sSf -o rustup-init.exe https://win.rustup.rs | ||
./rustup-init.exe -y --default-toolchain ${{ matrix.rustup_toolchain }} --default-host x86_64-pc-windows-msvc | ||
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin" | ||
if: ${{ startsWith(matrix.os, 'windows') }} | ||
hegerdes marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- name: Cargo build (Native TLS) | ||
run: cargo build --all --no-default-features --features=native-tls && cargo clean | ||
|
||
- name: Cargo build (Rust TLS) | ||
run: cargo build --all | ||
|
||
- name: Cargo test | ||
run: cargo test --all | ||
|
||
- name: Cargo fmt | ||
run: cargo fmt --check |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
name: Release | ||
|
||
on: | ||
push: | ||
tags: ["v*.*.*"] | ||
|
||
env: | ||
REGISTRY: ghcr.io | ||
IMAGE_NAME: ${{ github.repository }} | ||
# Cross-compilation for aarch64 requires a different linker | ||
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc | ||
|
||
permissions: | ||
contents: read | ||
|
||
jobs: | ||
Release-Build: | ||
runs-on: ${{ matrix.os }} | ||
permissions: | ||
contents: read | ||
attestations: write | ||
id-token: write | ||
strategy: | ||
matrix: | ||
target: | ||
- x86_64-unknown-linux-gnu | ||
- aarch64-unknown-linux-gnu | ||
- x86_64-pc-windows-msvc | ||
- x86_64-apple-darwin | ||
- aarch64-apple-darwin | ||
rustup_toolchain: [stable] | ||
include: | ||
- os: windows-2022 | ||
target: x86_64-pc-windows-msvc | ||
- os: ubuntu-20.04 | ||
target: x86_64-unknown-linux-gnu | ||
- os: ubuntu-20.04 | ||
target: aarch64-unknown-linux-gnu | ||
- os: macos-13 | ||
target: x86_64-apple-darwin | ||
- os: macos-14 | ||
target: aarch64-apple-darwin | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install Rust (UNIX) | ||
if: ${{ ! startsWith(matrix.os, 'windows') }} | ||
run: | | ||
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain ${{ matrix.rustup_toolchain }} | ||
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin" | ||
# We need to crosscompile for arm64 linux for now | ||
if [[ "${{ matrix.target }}" == "aarch64-unknown-linux-gnu" ]]; then | ||
echo "Adding aarch64-unknown-linux-gnu target" | ||
sudo apt-get update -y | ||
sudo apt-get install -y make g++ libssl-dev gcc-aarch64-linux-gnu | ||
rustup target add aarch64-unknown-linux-gnu | ||
else | ||
echo "Compiling for ${{matrix.target}}" | ||
fi | ||
|
||
- name: Install Rust (Windows) | ||
if: ${{ startsWith(matrix.os, 'windows') }} | ||
run: | | ||
curl -sSf -o rustup-init.exe https://win.rustup.rs | ||
./rustup-init.exe -y --default-toolchain ${{ matrix.rustup_toolchain }} --default-host ${{ matrix.target }} | ||
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin" | ||
hegerdes marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- name: Cargo build | ||
run: cargo build --release --target ${{ matrix.target }} | ||
|
||
- name: Archive (UNIX) | ||
run: | | ||
mkdir -p artifacts | ||
cp -av target/${{ matrix.target }}/release/zola . | ||
tar -czf ${{ github.event.repository.name }}-${{ github.ref_name }}-${{ matrix.target }}.tar.gz zola | ||
if: ${{ ! startsWith(matrix.os, 'windows') }} | ||
|
||
- name: Archive (Windows) | ||
run: | | ||
mkdir -p artifacts | ||
cp target/${{ matrix.target }}/release/zola.exe . | ||
tar -czf ${{ github.event.repository.name }}-${{ github.ref_name }}-${{ matrix.target }}.zip zola.exe | ||
if: ${{ startsWith(matrix.os, 'windows') }} | ||
|
||
- name: Attest Build Provenance | ||
uses: actions/attest-build-provenance@v1 | ||
continue-on-error: true | ||
with: | ||
subject-path: ${{ github.event.repository.name }}-${{ github.ref_name }}-${{ matrix.target }}.* | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: ${{ github.event.repository.name }}-${{ github.ref_name }}-${{ matrix.target }} | ||
path: ${{ github.event.repository.name }}-${{ github.ref_name }}-${{ matrix.target }}.* | ||
if-no-files-found: error | ||
retention-days: 7 | ||
|
||
Release: | ||
needs: [Release-Build] | ||
runs-on: ubuntu-latest | ||
permissions: | ||
contents: write | ||
|
||
steps: | ||
- name: Ensure artifacts dir exists | ||
run: mkdir -p artifacts | ||
|
||
- name: Download Artifact | ||
uses: actions/download-artifact@v4 | ||
with: | ||
path: artifacts | ||
merge-multiple: true | ||
|
||
- name: Release | ||
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191 | ||
with: | ||
name: ${{ github.ref_name }} | ||
tag_name: ${{ github.ref_name }} | ||
generate_release_notes: true | ||
fail_on_unmatched_files: true | ||
body: | | ||
Welcome to this new release of Zola ${{ github.ref_name }}! | ||
|
||
All artifacts are signed with this repos identity using Sigstore. | ||
You can verify the signatures using the `GitHub` CLI. | ||
|
||
```shell | ||
gh attestation verify --owner ${{ github.repository_owner }} <my-artifact> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right now Azure generates a list of commits since the last release (https://github.com/getzola/zola/releases/tag/v0.19.2), is it possible to keep it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah it should generate release notes with I'm also using this action here: https://github.com/hegerdes/joplin-plugin-remote-note-pull/releases/tag/v1.3.0 and Pulumi also uses uses with some more changes than my small project, see https://github.com/pulumi/pulumi-awsx/releases I think they might look a litte different to azures notes though. Feel fee to add whatever you want to that release body you are the owner, it was just an example for now. |
||
``` | ||
token: ${{ secrets.GITHUB_TOKEN }} | ||
prerelease: ${{ contains(github.ref, '-pre') }} | ||
files: artifacts/* | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's the reason behind splitting the Release-Build and Release jobs? Why not do it in 1 job? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The build for each target run all on their own isolated runner. So the targets can be build native and in parallel. The runners don't share any storage so wee need to get all build assets on one runner. This is done via upload/download artifacts. The artifacts assets are temporary and will auto delete in one week. When each runner creates a release we can have race conditions or partly failed release jobs. This ensures the release build for every target worked and only then one release is created. If one target fails -> no release. The argument |
||
|
||
Release-Container-Image: | ||
needs: [Release] | ||
runs-on: ubuntu-latest | ||
permissions: | ||
contents: read | ||
packages: write | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf | ||
|
||
- name: Setup Docker buildx | ||
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db | ||
|
||
- name: Log in to the Container registry | ||
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9 | ||
with: | ||
registry: ${{ env.REGISTRY }} | ||
username: ${{ github.actor }} | ||
password: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
- name: Extract metadata (tags, labels) for Docker | ||
id: meta | ||
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38 | ||
with: | ||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
flavor: latest=false | ||
|
||
- name: Build and push Docker image | ||
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc | ||
with: | ||
context: . | ||
push: true | ||
build-args: USE_GH_RELEASE=true | ||
platforms: linux/amd64,linux/arm64 | ||
tags: ${{ steps.meta.outputs.tags }} | ||
labels: ${{ steps.meta.outputs.labels }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,21 @@ | ||
FROM rust:slim-bookworm AS builder | ||
|
||
RUN apt-get update -y && \ | ||
apt-get install -y make g++ libssl-dev && \ | ||
rustup target add x86_64-unknown-linux-gnu | ||
ARG USE_GH_RELEASE=false | ||
RUN apt-get update -y && apt-get install -y make g++ libssl-dev curl jq tar gzip | ||
|
||
WORKDIR /app | ||
COPY . . | ||
|
||
RUN cargo build --release --target x86_64-unknown-linux-gnu | ||
|
||
RUN if [ "${USE_GH_RELEASE}" = "true" ]; then \ | ||
mkdir -p target/$(uname -m)-unknown-linux-gnu/release && \ | ||
export ZOLA_VERSION=$(curl -sL https://api.github.com/repos/getzola/zola/releases/latest | jq -r .name) && \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not reproducible, can you pass the version to the Dockerfile instead? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Totally agree. Just wanted to check if this approach is fine before going all the way. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sounds good |
||
curl -sL --fail --output zola.tar.gz https://github.com/getzola/zola/releases/download/${ZOLA_VERSION}/zola-${ZOLA_VERSION}-$(uname -m)-unknown-linux-gnu.tar.gz && \ | ||
tar -xzvf zola.tar.gz zola; \ | ||
else \ | ||
cargo build --release && \ | ||
cp target/$(uname -m)-unknown-linux-gnu/release/zola zola; \ | ||
fi && ./zola --version | ||
|
||
FROM gcr.io/distroless/cc-debian12 | ||
COPY --from=builder /app/target/x86_64-unknown-linux-gnu/release/zola /bin/zola | ||
COPY --from=builder /app/zola /bin/zola | ||
ENTRYPOINT [ "/bin/zola" ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to run tests on different archs for the same OS?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GitHubs Action matrix is a little counter-intuitive. A job for each
target
(currently 5) and eachrustup_toolchain
(currently 1) is created. Theinclude
part sets the os. We use the native os for each target, except linux arm. I thought testing on the native arch is best when ever possible - but I know very litte about the rust build standards.If you think testing with cross compiling is better or also needed required let me know the exact test matrix.