Skip to content

Commit

Permalink
Build CLI for Linux ARM64 (#5503)
Browse files Browse the repository at this point in the history
### What

Add support for building a Linux ARM64 version of the CLI in our build
system.

- Follow-up to #5489
- Part of #4136

This PR introduces ugly work-around to be cleaned when our pixi deps
support linux-aarch64:
- #5507

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5503)
- [Docs
preview](https://rerun.io/preview/65c8082ad0c5482bad9167e8620490494f80637e/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/65c8082ad0c5482bad9167e8620490494f80637e/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

---------

Co-authored-by: Andreas Reich <andreas@rerun.io>
  • Loading branch information
abey79 and Wumpf authored Mar 14, 2024
1 parent 9977717 commit 56bb2ea
Show file tree
Hide file tree
Showing 4 changed files with 74 additions and 36 deletions.
46 changes: 28 additions & 18 deletions .github/workflows/on_push_main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -162,40 +162,49 @@ jobs:
# -----------------------------------------------------------------------------------
# Build rerun-cli (rerun binaries):

build-rerun-cli-and-upload-linux:
build-rerun-cli-and-upload-linux-arm64:
needs: [checks]
name: "Linux-arm64: Build & Upload rerun-cli"
uses: ./.github/workflows/reusable_build_and_upload_rerun_cli.yml
with:
CONCURRENCY: push-linux-arm64-${{ github.ref_name }}
PLATFORM: linux-arm64
secrets: inherit

build-rerun-cli-and-upload-linux-x64:
needs: [checks]
name: "Linux-x64: Build & Upload rerun-cli"
uses: ./.github/workflows/reusable_build_and_upload_rerun_cli.yml
with:
CONCURRENCY: push-linux-${{ github.ref_name }}
PLATFORM: linux
CONCURRENCY: push-linux-x64-${{ github.ref_name }}
PLATFORM: linux-x64
secrets: inherit

build-rerun-cli-and-upload-macos-intel:
build-rerun-cli-and-upload-macos-x64:
needs: [checks]
name: "Mac-Intel: Build & Upload rerun-cli"
name: "Mac-x64: Build & Upload rerun-cli"
uses: ./.github/workflows/reusable_build_and_upload_rerun_cli.yml
with:
CONCURRENCY: push-macos-intel-${{ github.ref_name }}
PLATFORM: macos-intel
CONCURRENCY: push-macos-x64-${{ github.ref_name }}
PLATFORM: macos-x64
secrets: inherit

build-rerun-cli-and-upload-macos-arm:
build-rerun-cli-and-upload-macos-arm64:
needs: [checks]
name: "Mac-Arm: Build & Upload rerun-cli"
name: "Mac-arm64: Build & Upload rerun-cli"
uses: ./.github/workflows/reusable_build_and_upload_rerun_cli.yml
with:
CONCURRENCY: push-macos-arm-${{ github.ref_name }}
PLATFORM: macos-arm
CONCURRENCY: push-macos-arm64-${{ github.ref_name }}
PLATFORM: macos-arm64
secrets: inherit

build-rerun-cli-and-upload-windows:
build-rerun-cli-and-upload-windows-x64:
needs: [checks]
name: "Windows-x64: Build & Upload rerun-cli"
uses: ./.github/workflows/reusable_build_and_upload_rerun_cli.yml
with:
CONCURRENCY: push-windows-${{ github.ref_name }}
PLATFORM: windows
CONCURRENCY: push-windows-x64-${{ github.ref_name }}
PLATFORM: windows-x64
secrets: inherit

# -----------------------------------------------------------------------------------
Expand Down Expand Up @@ -305,10 +314,11 @@ jobs:
build-rerun_c-and-upload-macos-x64,
build-rerun_c-and-upload-macos-arm64,
build-rerun_c-and-upload-windows-x64,
build-rerun-cli-and-upload-linux,
build-rerun-cli-and-upload-macos-intel,
build-rerun-cli-and-upload-macos-arm,
build-rerun-cli-and-upload-windows,
build-rerun-cli-and-upload-linux-x64,
build-rerun-cli-and-upload-linux-arm64,
build-rerun-cli-and-upload-macos-x64,
build-rerun-cli-and-upload-macos-arm64,
build-rerun-cli-and-upload-windows-x64,
bundle-and-upload-rerun_cpp,
]
runs-on: "ubuntu-latest"
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/reusable_build_and_upload_rerun_c.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ on:
required: false
type: string
default: "adhoc"
description: "Concurrency group to use"

concurrency:
group: ${{ inputs.CONCURRENCY }}-build-rerun_c
Expand Down
39 changes: 29 additions & 10 deletions .github/workflows/reusable_build_and_upload_rerun_cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,25 @@ on:
workflow_dispatch:
inputs:
ADHOC_NAME:
required: true
required: false
type: string
default: ""
description: "Name of the adhoc build, used for upload directory"
PLATFORM:
type: choice
options:
- linux
- windows
- macos-arm
- macos-intel
- linux-arm64
- linux-x64
- windows-x64
- macos-arm64
- macos-x64
description: "Platform to build for"
required: true
CONCURRENCY:
required: false
type: string
default: "adhoc"
description: "Concurrency group to use"

concurrency:
group: ${{ inputs.CONCURRENCY }}-build-rerun-cli
Expand Down Expand Up @@ -71,25 +74,31 @@ jobs:
shell: bash
run: |
case "${{ inputs.PLATFORM }}" in
linux)
linux-arm64)
runner="buildjet-8vcpu-ubuntu-2204-arm"
target="aarch64-unknown-linux-gnu"
container="null"
bin_name="rerun"
;;
linux-x64)
runner="ubuntu-latest-16-cores"
target="x86_64-unknown-linux-gnu"
container="{'image': 'rerunio/ci_docker:0.11.0'}"
bin_name="rerun"
;;
windows)
windows-x64)
runner="windows-latest-8-cores"
target="x86_64-pc-windows-msvc"
container="null"
bin_name="rerun.exe"
;;
macos-arm)
macos-arm64)
runner="macos-latest-large" # See https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/
target="aarch64-apple-darwin"
container="null"
bin_name="rerun"
;;
macos-intel)
macos-x64)
runner="macos-latest-large" # See https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/
target="x86_64-apple-darwin"
container="null"
Expand Down Expand Up @@ -138,16 +147,26 @@ jobs:
targets: ${{ needs.set-config.outputs.TARGET }}

- uses: prefix-dev/setup-pixi@v0.4.1
if: ${{ inputs.PLATFORM != 'linux-arm64' }} # TODO(#5507): see below
with:
pixi-version: v0.13.0

- name: Build web-viewer (release)
if: ${{ inputs.PLATFORM != 'linux-arm64' }} # TODO(#5507): see below
shell: bash
run: |
pixi run cargo run --locked -p re_build_web_viewer -- --release
# TODO(#5507): supress this workaround when pixi dependencies are available on linux-arm64
- name: Build web-viewer (release, Linux ARM64)
if: ${{ inputs.PLATFORM == 'linux-arm64' }}
shell: bash
run: |
sudo apt-get install binaryen
cargo run --locked -p re_build_web_viewer -- --release
# This does not run in the pixi environment, doing so
# causes it to select the wrong compiler on macos-arm
# causes it to select the wrong compiler on macos-arm64
- name: Build rerun-cli
shell: bash
env:
Expand Down
24 changes: 16 additions & 8 deletions scripts/ci/sync_release_assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,19 +96,23 @@ def fetch_binary_assets(
rerun_c_blobs = [
(
f"rerun_c-{tag}-x86_64-pc-windows-msvc.lib",
f"commit/{commit_short}/rerun_c/windows/rerun_c.lib",
f"commit/{commit_short}/rerun_c/windows-x64/rerun_c.lib",
),
(
f"librerun_c-{tag}-x86_64-unknown-linux-gnu.a",
f"commit/{commit_short}/rerun_c/linux/librerun_c.a",
f"commit/{commit_short}/rerun_c/linux-x64/librerun_c.a",
),
(
f"librerun_c-{tag}-aarch64-unknown-linux-gnu.a",
f"commit/{commit_short}/rerun_c/linux-arm64/librerun_c.a",
),
(
f"librerun_c-{tag}-aarch64-apple-darwin.a",
f"commit/{commit_short}/rerun_c/macos-arm/librerun_c.a",
f"commit/{commit_short}/rerun_c/macos-arm64/librerun_c.a",
),
(
f"librerun_c-{tag}-x86_64-apple-darwin.a",
f"commit/{commit_short}/rerun_c/macos-intel/librerun_c.a",
f"commit/{commit_short}/rerun_c/macos-x64/librerun_c.a",
),
]
for name, blob_url in rerun_c_blobs:
Expand Down Expand Up @@ -139,19 +143,23 @@ def fetch_binary_assets(
rerun_cli_blobs = [
(
f"rerun-cli-{tag}-x86_64-pc-windows-msvc.exe",
f"commit/{commit_short}/rerun-cli/windows/rerun.exe",
f"commit/{commit_short}/rerun-cli/windows-x64/rerun.exe",
),
(
f"rerun-cli-{tag}-x86_64-unknown-linux-gnu",
f"commit/{commit_short}/rerun-cli/linux/rerun",
f"commit/{commit_short}/rerun-cli/linux-x64/rerun",
),
(
f"rerun-cli-{tag}-aarch64-unknown-linux-gnu",
f"commit/{commit_short}/rerun-cli/linux-arm64/rerun",
),
(
f"rerun-cli-{tag}-aarch64-apple-darwin",
f"commit/{commit_short}/rerun-cli/macos-arm/rerun",
f"commit/{commit_short}/rerun-cli/macos-arm64/rerun",
),
(
f"rerun-cli-{tag}-x86_64-apple-darwin",
f"commit/{commit_short}/rerun-cli/macos-intel/rerun",
f"commit/{commit_short}/rerun-cli/macos-x64/rerun",
),
]
for name, blob_url in rerun_cli_blobs:
Expand Down

0 comments on commit 56bb2ea

Please sign in to comment.