Skip to content

Commit

Permalink
Rollup merge of rust-lang#125669 - ferrocene:hoverbear/ci-docker-risc…
Browse files Browse the repository at this point in the history
…v64gc-update, r=Kobzol

CI: Update riscv64gc-linux job to Ubuntu 22.04, rename to riscv64gc-gnu

Together with joshua.zivkovic@codethink.co.uk, we've been starting to explore improving the state of the `riscv64gc-unknown-linux-gnu` target. Additionally, I'm looking to add support for this platform in [Ferrocene](https://github.com/ferrocene/ferrocene) ([Related PR](ferrocene/ferrocene#618)).

There currently exists a `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` job for the CI, however it is currently experiencing errors.

<details>

<summary>Errors</summary>

```bash
$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-linux
# ...
[RUSTC-TIMING] addr2line test:false 0.371
[RUSTC-TIMING] gimli test:false 3.159
[RUSTC-TIMING] object test:false 4.249
error: linking with `riscv64-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "riscv64-linux-gnu-gcc" "-Wl,--version-script=/tmp/rustcQaIpWi/list" "-Wl,--no-undefined-version" "/tmp/rustcQaIpWi/symbols.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.std.1b5d59225ff40bd2-cgu.0.rcgu.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.dalhl7sfna1ffn4nhy6pyfa7f.rcgu.rmeta" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.ef0znsdf1ihn2bjkmclodhclp.rcgu.o" "-Wl,--as-needed" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/build/compiler_builtins-9e9a40064e2f2bd3/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libpanic_unwind-d968371aba64a26c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libobject-da5b6473912e89d6.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libmemchr-9cfa08d2baa3643e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libaddr2line-06e0d2153cecb6ce.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libgimli-6fdf5551cec83840.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_demangle-8ada6466f763fa2e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd_detect-edc0d12d029c4c86.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libhashbrown-9c782935934c8c14.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_alloc-b6984e43b381efa4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libminiz_oxide-37ee29bf49ccaa96.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libadler-591133f6804fa0f4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libunwind-94d98075f42175f3.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcfg_if-e267a7b9dd7af3a7.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liblibc-503571a038f8d9fd.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liballoc-e36c72a5cf0ee45f.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_core-076c2b8501e25f03.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcore-c446fff80486d0bb.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd-ff89a9732cd5d858.so" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
# ...
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.124.rcgu.o)
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: -march=rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0: unsupported ISA subset `z'
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.004.rcgu.o)
          collect2: error: ld returned 1 exit status

[RUSTC-TIMING] std test:false 15.138
error: could not compile `std` (lib) due to 1 previous error
Building bootstrap
Build completed unsuccessfully in 0:04:41
  local time: Tue May 28 16:25:09 UTC 2024
  network time: Tue, 28 May 2024 16:25:17 GMT
```

</details>

This PR fixes the breakage enough to get the tests running. It does so through bringing the `riscv64gc-unknown-linux-gnu` related test job in line with other related jobs, adopting many of the recent changes present in `src/ci/docker/host-x86_64/armhf-gnu` such as:

* Using Ubuntu 22.04
* Installing a more narrowly scoped package set
* Using `curl` instead of `debootstrap` to set up the root
* No longer patching `busybox`
* Removing the `cmake.sh` script related steps

## Justifying Renaming `riscv64gc-linux` to `riscv64gc-gnu`

The `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` job runs the tests for `risv64gc-unknown-linux-gnu`, it is based off `src/ci/docker/host-x86_64/armhf-gnu`.

There are other jobs that follow a `$arch-gnu` naming scheme:

* `src/ci/docker/host-x86_64/armhf-gnu`
* `src/ci/docker/host-x86_64/x86_64-gnu`
* `src/ci/docker/host-aarch64/aarch64-gnu`

It follows that the name `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` should be `src/ci/docker/host-x86_64/disabled/riscv64gc-gnu`, like the others.

## Testing

> [!NOTE]
> `riscv64gc-unknown-linux-gnu` is a [**Tier 2 with Host Tools** platform](https://doc.rust-lang.org/beta/rustc/platform-support.html), all tests may not necessarily pass! There is work in rust-lang#125220 which helps fix several related tests.

You can test out the renamed job:

```sh
DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
```

`DEPLOY=1` helps reproduce the CI's environment and also avoids the chance of a `llvm-c/BitReader.h` error (detailed in rust-lang#85424 and rust-lang#56650).

<details>

<summary>Sample of output (expected test failure)</summary>

```bash
$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
# ...
test [ui] tests/ui/where-clauses/where-clause-method-substituion-rpass.rs ... ok

failures:

---- [ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=riscv64gc-unknown-linux-gnu" "--check-cfg" "cfg(FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo" "-A" "unused" "-A" "internal_features" "-Crpath" "-Lnative=/checkout/obj/build/riscv64gc-unknown-linux-gnu/native/rust-test-helpers" "-Clinker=riscv64-linux-gnu-gcc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo/auxiliary" "-g" "--emit=llvm-ir" "-Csplit-debuginfo=unpacked"
stdout: none
--- stderr -------------------------------
error: `-Csplit-debuginfo=unpacked` is unstable on this platform

error: aborting due to 1 previous error
------------------------------------------

failures:
    [ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs

test result: FAILED. 5 passed; 1 failed; 16897 ignored; 0 measured; 0 filtered out; finished in 410.99ms

Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=riscv64gc-unknown-linux-gnu
  local time: Tue May 28 16:28:22 UTC 2024
  network time: Tue, 28 May 2024 16:28:30 GMT
```

</details>

try-job: riscv64gc-gnu
  • Loading branch information
jieyouxu authored Jun 11, 2024
2 parents ecc0046 + e1b489d commit 91374fa
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 148 deletions.
Original file line number Diff line number Diff line change
@@ -1,39 +1,36 @@
# based on armhf-gnu/Dockerfile
FROM ubuntu:20.04
FROM ubuntu:22.04

RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update -y && apt-get install -y --no-install-recommends \
bc \
bison \
ca-certificates \
cmake \
cpio \
curl \
debian-ports-archive-keyring \
debootstrap \
flex \
gcc \
gcc-riscv64-linux-gnu \
git \
g++-riscv64-linux-gnu \
g++ \
libc6-dev \
libc6-dev-riscv64-cross \
libssl-dev \
make \
ninja-build \
patch \
python3 \
qemu-system-misc \
xz-utils
bc \
bzip2 \
ca-certificates \
cmake \
cpio \
curl \
file \
flex \
bison \
g++ \
g++-riscv64-linux-gnu \
git \
libc6-dev \
libc6-dev-riscv64-cross \
libssl-dev \
make \
ninja-build \
python3 \
qemu-system-riscv64 \
xz-utils

ENV ARCH=riscv
ENV CROSS_COMPILE=riscv64-linux-gnu-
ENV ARCH=riscv \
CROSS_COMPILE=riscv64-linux-gnu-

WORKDIR /build

# From https://github.com/michaeljclark/busybear-linux/blob/master/conf/linux.config
COPY host-x86_64/riscv64gc-linux/linux.config /build
COPY host-x86_64/riscv64gc-gnu/linux.config /build

# Compile the kernel that we're going to be emulating with. This is
# basically just done to be compatible with the QEMU target that we're going
Expand All @@ -49,29 +46,22 @@ RUN curl https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.16.tar.xz | tar
# Compile an instance of busybox as this provides a lightweight system and init
# binary which we will boot into. Only trick here is configuring busybox to
# build static binaries.
RUN curl https://busybox.net/downloads/busybox-1.31.1.tar.bz2 | tar xjf -
COPY host-x86_64/riscv64gc-linux/0001-Remove-stime-function-calls.patch /build/busybox-1.31.1/
RUN cd /build/busybox-1.31.1 && \
patch -p1 -i 0001-Remove-stime-function-calls.patch && \
make defconfig && \
sed -i 's/.*CONFIG_STATIC.*/CONFIG_STATIC=y/' .config && \
make -j$(nproc) && \
make install && \
mv _install /tmp/rootfs && \
cd /build && \
rm -rf busybox-1.31.1
RUN curl https://www.busybox.net/downloads/busybox-1.32.1.tar.bz2 | tar xjf - && \
cd busybox-1.32.1 && \
make defconfig && \
sed -i 's/.*CONFIG_STATIC.*/CONFIG_STATIC=y/' .config && \
make -j$(nproc) && \
make install && \
mv _install /tmp/rootfs && \
cd /build && \
rm -rf busybox-1.32.1

# Download the ubuntu rootfs, which we'll use as a chroot for all our tests
# This is only needed to provide /lib/* and /usr/lib/*
# Download the ubuntu rootfs, which we'll use as a chroot for all our tests.
WORKDIR /tmp
RUN debootstrap --variant=minbase --arch=riscv64 --foreign focal /tmp/rootfs/ubuntu
RUN cd rootfs && mkdir proc sys dev etc etc/init.d
# rootfs/ubuntu/proc is in a weird state (access fails with ELOOP) until
# rootfs/ubuntu/debootstrap/debootstrap --second-stage is run (under emulation),
# but this takes ages. Instead hack it into a good enough state.
# /proc is used by std::env::current_exe() (which is roughly
# `readlink /proc/self/exe`)
RUN cd rootfs/ubuntu && rm -rf proc && mkdir proc
RUN mkdir rootfs/ubuntu
RUN curl https://cdimage.ubuntu.com/ubuntu-base/releases/22.04/release/ubuntu-base-22.04.2-base-riscv64.tar.gz | \
tar xzf - -C rootfs/ubuntu && \
cd rootfs && mkdir proc sys dev etc etc/init.d

# Copy over our init script, which starts up our test server and also a few other
# misc tasks
Expand All @@ -95,12 +85,12 @@ RUN mkdir build && cd build && \
WORKDIR /tmp
RUN rm -rf /tmp/riscv-pk

COPY scripts/cmake.sh /scripts/
RUN /scripts/cmake.sh

COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

# Avoid "fatal: detected dubious ownership in repository at '/checkout'" error
RUN git config --global --add safe.directory /checkout

ENV RUST_CONFIGURE_ARGS --qemu-riscv64-rootfs=/tmp/rootfs
ENV SCRIPT python3 ../x.py --stage 2 test --host='' --target riscv64gc-unknown-linux-gnu

Expand Down

This file was deleted.

0 comments on commit 91374fa

Please sign in to comment.