How to build a multi platform aot docker image

[WeihanLi]

I'm trying to build a multi-platform supported docker image with aot publish, but while can not make it work for multi-platform build, get some errors like the following

linker command failed with exit code 1

 > [linux/amd64->arm64 build-env 11/11] RUN dotnet publish -f net9.0 --use-current-runtime -a arm64 -p:AssemblyName=http -p:TargetFrameworks=net9.0 -o /app/artifacts:
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lgcc: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lgcc_s: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lpthread: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lc: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lgcc: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find -lgcc_s: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find crtendS.o: No such file or directory
115.8   /usr/bin/aarch64-linux-gnu-ld.bfd: cannot find crtn.o: No such file or directory
115.8 clang : error : linker command failed with exit code 1 (use -v to see invocation) [/app/src/HTTPie/HTTPie.csproj::TargetFramework=net9.0]
115.8 /root/.nuget/packages/microsoft.dotnet.ilcompiler/9.0.0/build/Microsoft.NETCore.Native.targets(376,5): error MSB3073: The command ""clang" "obj/Release/net9.0/linux-arm64/native/http.o" -o "bin/Release/net9.0/linux-arm64/native/http" -Wl,--version-script=obj/Release/net9.0/linux-arm64/native/http.exports -Wl,--export-dynamic -gz=zlib -fuse-ld=bfd /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libbootstrapper.o /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libRuntime.WorkstationGC.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libeventpipe-disabled.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libstandalonegc-disabled.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libstdc++compat.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Native.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.IO.Compression.Native.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Net.Security.Native.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Security.Cryptography.Native.OpenSsl.a /root/.nuget/packages/runtime.linux-arm64.microsoft.dotnet.ilcompiler/9.0.0/sdk/libz.a --target=aarch64-linux-gnu -g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -ldl -lrt -lm -pie -Wl,-pie -Wl,-z,relro -Wl,-z,now -Wl,--eh-frame-hdr -Wl,--discard-all -Wl,--gc-sections" exited with code 1. [/app/src/HTTPie/HTTPie.csproj::TargetFramework=net9.0]

the Dockerfile likes below:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:9.0 AS build-env
ARG TARGETARCH

# Configure NativeAOT Build Prerequisites 
# https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=linux-alpine%2Cnet8
# for alpine
# RUN apk update && apk add clang build-base zlib-dev
# for debian/ubuntu
# https://github.com/dotnet/runtimelab/issues/1785#issuecomment-993179119
RUN apt-get update && apt-get install -y clang zlib1g-dev binutils-aarch64-linux-gnu

WORKDIR /app

COPY ./src/ ./src/
COPY ./build/ ./build/
COPY ./Directory.Build.props ./
COPY ./Directory.Build.targets ./
COPY ./Directory.Packages.props ./
COPY ./.editorconfig ./

WORKDIR /app/src/HTTPie/
RUN dotnet publish -f net9.0 --use-current-runtime -a $TARGETARCH -p:AssemblyName=http -p:TargetFrameworks=net9.0 -o /app/artifacts

FROM scratch

# https://github.com/opencontainers/image-spec/blob/main/annotations.md
LABEL org.opencontainers.image.authors="WeihanLi"
LABEL org.opencontainers.image.source="https://github.com/WeihanLi/dotnet-httpie"

WORKDIR /app
COPY --from=build-env /app/artifacts/http /app/http
ENV PATH="/app:${PATH}"
ENTRYPOINT ["/app/http"]
CMD ["--help"]

Dockerfile: https://github.com/WeihanLi/dotnet-httpie/blob/cddc41ed5f26f57e2c5c56d2996d861a00ad44cd/Dockerfile
Azure DevOps Pipeline config: https://github.com/WeihanLi/dotnet-httpie/blob/cddc41ed5f26f57e2c5c56d2996d861a00ad44cd/.azure/pipelines/docker.yml

Is there a sample Dockerfile for building an AOT publish multi-platform container image?

[am11]

Dockerfile: WeihanLi/dotnet-httpie@cddc41e/Dockerfile

Here is the diff: https://gist.github.com/am11/d2519126fb9bc6665f9b4ececbb3d411/revisions

that additional file command at the end prints:

/app/artifacts/http: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=8e8a12134dd8a5ba09be97a1ef2e7447406447fd, for GNU/Linux 3.2.0, stripped
...
/app/artifacts/http: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=f4a9ba02c1e46fa5081b454f6b62821285afe6a3, for GNU/Linux 3.7.0, stripped

confirms published binary has intended arch.

Azure DevOps Pipeline config: WeihanLi/dotnet-httpie@cddc41e/.azure/pipelines/docker.yml

In pipeline yml file, you may want use docker run --rm --privileged multiarch/qemu-user-static --reset -p yes instead of tonistiigi/binfmt line.

[WeihanLi]

Thanks very much for the help, that saved me a lot.

Seemed when I updated the tonistiigi/binfmt line to docker run --rm --privileged multiarch/qemu-user-static --reset -p yes, got an error likes docker exporter does not currently support exporting manifest lists, while I see it success in your build and the config seems to be the same

build history:
https://weihanli.visualstudio.com/Pipelines/_build/results?buildId=6806&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=5caf77c8-9b10-50ef-b5c7-ca89c63e1c86

[am11]

I tested it here: https://dev.azure.com/adeelbm/adeelbm/_build/results?buildId=21&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=5caf77c8-9b10-50ef-b5c7-ca89c63e1c86

Dockerfile: https://github.com/am11/dotnet-httpie/blob/e900cb05978354771e13edfc0f1bd9ccd128f4e3/Dockerfile
docker.yml: https://github.com/am11/dotnet-httpie/blob/e900cb05978354771e13edfc0f1bd9ccd128f4e3/.azure/pipelines/docker.yml

in your case, this is the closest one: https://weihanli.visualstudio.com/Pipelines/_build/results?buildId=6810&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=5caf77c8-9b10-50ef-b5c7-ca89c63e1c86

You can revert to WeihanLi/dotnet-httpie@048155a and add docker login step (before buildx step):

- script: |
    docker login -u $(dockerId) -p $(pswd)
  displayName: 'Docker login'

[WeihanLi]

Thank you very much ❤️
Based on your help, I managed to move it to the alpine-based image also, many thanks