Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gateway to snapshot builds in CI #3256

Merged
merged 1 commit into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/snapshot-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ jobs:
- image: node
base-artifact: subspace-node
upload-executables: true
- image: gateway
base-artifact: subspace-gateway
upload-executables: false
- image: bootstrap-node
base-artifact: subspace-bootstrap-node
upload-executables: false
Expand Down Expand Up @@ -104,7 +107,6 @@ jobs:
cd executables
IMAGE="${{ fromJSON(steps.meta.outputs.json).tags[0] }}"
ARTIFACT="${{ matrix.build.base-artifact }}"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It set off my editor's trailing spaces auto-formatting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing trailing spaces is good, but this one got the whole line itself removed. It is fine.

docker run --rm --platform linux/amd64 --entrypoint /bin/cat $IMAGE /$ARTIFACT > $ARTIFACT-ubuntu-x86_64-skylake-${{ github.ref_name }}
# TODO: Pull is a workaround for https://github.com/moby/moby/issues/48197#issuecomment-2472265028
docker pull --platform linux/amd64/v2 $IMAGE
Expand Down
109 changes: 109 additions & 0 deletions docker/gateway.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# This Dockerfile supports both native building and cross-compilation to x86-64, aarch64 and riscv64
FROM --platform=$BUILDPLATFORM ubuntu:22.04

ARG RUSTC_VERSION=nightly-2024-10-22
ARG PROFILE=production
ARG RUSTFLAGS
# Incremental compilation here isn't helpful
ENV CARGO_INCREMENTAL=0
ENV PKG_CONFIG_ALLOW_CROSS=true

ARG BUILDARCH
ARG TARGETARCH

WORKDIR /code

RUN \
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates \
protobuf-compiler \
curl \
git \
llvm \
clang \
automake \
libtool \
pkg-config \
make

RUN \
if [ $BUILDARCH != "arm64" ] && [ $TARGETARCH = "arm64" ]; then \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
g++-aarch64-linux-gnu \
gcc-aarch64-linux-gnu \
libc6-dev-arm64-cross \
; fi

RUN \
if [ $BUILDARCH != "riscv64" ] && [ $TARGETARCH = "riscv64" ]; then \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
g++-riscv64-linux-gnu \
gcc-riscv64-linux-gnu \
libc6-dev-riscv64-cross \
; fi

RUN \
if [ $BUILDARCH != "amd64" ] && [ $TARGETARCH = "amd64" ]; then \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
g++-x86-64-linux-gnu \
gcc-x86-64-linux-gnu \
libc6-dev-amd64-cross \
; fi

RUN \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain $RUSTC_VERSION && \
/root/.cargo/bin/rustup target add wasm32-unknown-unknown

COPY Cargo.lock /code/Cargo.lock
COPY Cargo.toml /code/Cargo.toml
COPY rust-toolchain.toml /code/rust-toolchain.toml

COPY crates /code/crates
COPY domains /code/domains
COPY shared /code/shared
COPY test /code/test

# Up until this line all Rust images in this repo should be the same to share the same layers

ARG TARGETVARIANT

RUN \
if [ $BUILDARCH != "arm64" ] && [ $TARGETARCH = "arm64" ]; then \
export RUSTFLAGS="$RUSTFLAGS -C linker=aarch64-linux-gnu-gcc" \
; fi && \
if [ $BUILDARCH != "riscv64" ] && [ $TARGETARCH = "riscv64" ]; then \
export RUSTFLAGS="$RUSTFLAGS -C linker=riscv64-linux-gnu-gcc" \
; fi && \
if [ $TARGETARCH = "amd64" ] && [ "$RUSTFLAGS" = "" ]; then \
case "$TARGETVARIANT" in \
# x86-64-v2 with AES-NI
"v2") export RUSTFLAGS="-C target-cpu=x86-64-v2" ;; \
# x86-64-v3 with AES-NI
"v3") export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+aes" ;; \
# v4 is compiled for Zen 4+
"v4") export RUSTFLAGS="-C target-cpu=znver4" ;; \
# Default build is for Skylake
*) export RUSTFLAGS="-C target-cpu=skylake" ;; \
esac \
; fi && \
if [ $BUILDARCH != "amd64" ] && [ $TARGETARCH = "amd64" ]; then \
export RUSTFLAGS="$RUSTFLAGS -C linker=x86_64-linux-gnu-gcc" \
; fi && \
RUSTC_TARGET_ARCH=$(echo $TARGETARCH | sed "s/amd64/x86_64/g" | sed "s/arm64/aarch64/g" | sed "s/riscv64/riscv64gc/g") && \
/root/.cargo/bin/cargo -Zgitoxide -Zgit build \
--locked \
-Z build-std \
--profile $PROFILE \
--bin subspace-gateway \
--target $RUSTC_TARGET_ARCH-unknown-linux-gnu && \
mv target/*/*/subspace-gateway subspace-gateway && \
rm -rf target

FROM ubuntu:22.04

COPY --from=0 /code/subspace-gateway /subspace-gateway

USER nobody:nogroup

ENTRYPOINT ["/subspace-gateway"]
1 change: 1 addition & 0 deletions docker/gateway.Dockerfile.dockerignore
Loading