Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Got error undefined reference to 'getauxval' during the compilation on aarch64 with musl #3324

Closed
janvier-vip opened this issue Apr 18, 2023 · 26 comments
Labels
bug not-rustup Whatever is described in this issue isn't Rustup's fault

Comments

@janvier-vip
Copy link

janvier-vip commented Apr 18, 2023

Problem

I am trying to build Rust application on aarch64 platform with musl.

After setting up the Docker, I started a container with the latest Alpine image and then installed the Rust and essential components as follows:

# apk update
# apk add build-base curl perl libgcc
# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# source "$HOME/.cargo/env"
# cargo -V
cargo 1.69.0 (6e9a83356 2023-04-12)

Everything looked good. The next step I prepared the main program:

main.rs

use std::error::Error;
use reqwest::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let client = Client::new();
    let response = client.get("https://www.example.com").send().await?;
    let body = response.text().await?;
    println!("Body:\n{}", body);
    Ok(())
}

Cargo.toml

[package]
name = "hello-reqwest"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
reqwest = "0.11.16"
tokio = { version = "1.27.0", features = ["full"] }
openssl = { version = "0.10.50", features = ["vendored"] }

Then started the building procedure and got the output as follows:

# cargo build
   Compiling autocfg v1.1.0
   Compiling libc v0.2.142
   Compiling cfg-if v1.0.0
   Compiling proc-macro2 v1.0.56
   Compiling unicode-ident v1.0.8
   Compiling quote v1.0.26
   Compiling log v0.4.17
   Compiling cc v1.0.79
   Compiling openssl-src v111.25.3+1.1.1t
   Compiling syn v2.0.15
   Compiling lock_api v0.4.9
   Compiling once_cell v1.17.1
   Compiling pin-project-lite v0.2.9
   Compiling pkg-config v0.3.26
   Compiling parking_lot_core v0.9.7
   Compiling futures-core v0.3.28
   Compiling scopeguard v1.1.0
   Compiling smallvec v1.10.0
   Compiling bytes v1.4.0
   Compiling openssl-sys v0.9.87
   Compiling tokio v1.28.0
   Compiling parking_lot v0.12.1
   Compiling signal-hook-registry v1.4.1
   Compiling mio v0.8.6
   Compiling socket2 v0.4.9
   Compiling num_cpus v1.15.0
   Compiling futures-task v0.3.28
   Compiling itoa v1.0.6
   Compiling tracing-core v0.1.30
   Compiling indexmap v1.9.3
   Compiling slab v0.4.8
   Compiling openssl v0.10.52
   Compiling fnv v1.0.7
   Compiling futures-util v0.3.28
   Compiling tokio-macros v2.1.0
   Compiling foreign-types-shared v0.1.1
   Compiling foreign-types v0.3.2
   Compiling openssl-macros v0.1.1
   Compiling http v0.2.9
   Compiling tracing v0.1.37
   Compiling bitflags v1.3.2
   Compiling futures-channel v0.3.28
   Compiling httparse v1.8.0
   Compiling futures-sink v0.3.28
   Compiling hashbrown v0.12.3
   Compiling pin-utils v0.1.0
   Compiling tinyvec_macros v0.1.1
   Compiling native-tls v0.2.11
   Compiling tinyvec v1.6.0
   Compiling try-lock v0.2.4
   Compiling percent-encoding v2.2.0
   Compiling openssl-probe v0.1.5
   Compiling serde v1.0.160
   Compiling form_urlencoded v1.1.0
   Compiling want v0.3.0
   Compiling unicode-normalization v0.1.22
   Compiling http-body v0.4.5
   Compiling httpdate v1.0.2
   Compiling tokio-util v0.7.8
   Compiling tower-service v0.3.2
   Compiling unicode-bidi v0.3.13
   Compiling h2 v0.3.18
   Compiling idna v0.3.0
   Compiling hyper v0.14.26
   Compiling ryu v1.0.13
   Compiling serde_urlencoded v0.7.1
   Compiling url v2.3.1
   Compiling encoding_rs v0.8.32
   Compiling base64 v0.21.0
   Compiling mime v0.3.17
   Compiling ipnet v2.7.2
   Compiling tokio-native-tls v0.3.1
   Compiling hyper-tls v0.5.0
   Compiling reqwest v0.11.17
   Compiling hello-reqwest v0.1.0 (/root/hello-request)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/bin:/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/bin/self-contained:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "cc" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtbegin.o" "/tmp/rustcjhJxP0/symbols.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.12uba3z9ighejiqs.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.145tyn9nbx7n270s.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.19fag5lc93ppob3c.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1d3fae1v8fhi5jmz.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1jd0oz8oqowd690z.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1ld7cl6ufrhow3a9.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1oqgb310o0uzfaux.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1rdn52n3ggbqlqlz.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1vlz05jsq7cilv64.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1xg2kpcta6la7cp7.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.20iccn5lbj9hna18.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2c37upk3ojtv5ewd.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2dqxif0axib2qxdg.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2fm2xs4sivytxh8h.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2hxto55wx3bgq8hd.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2id3zteqrharr6yy.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2jbchzsnr88rgfcz.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2otxrcztbitvem5.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.2xoy6yqn1g9mtxhd.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.30cs3vz9jk4mpng1.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.30m1l6w90g0dlr94.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.31ghvs0pya9eito4.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.32bqd75sv1puzjhi.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.345c00zqyhpp7r3r.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.3c3mbcvk731a2wb2.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.3hbpoay9ro5cuypk.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.3p36xl10pi2hx70h.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.3qtoyffcf9kr8tew.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.3xv3z1saqcm9xwcz.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.4bp1kophituia2qb.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.4glh4fkb0xrdfyi.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.4h1ltcfqsyites93.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.4x3jars6iwesr2xf.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.4z8vwlcc4rheb0bt.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.5g8z9cgyiv4vqhlr.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.aw94dkee94gdbmf.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.dyyewhsitf84089.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.ic48d3plvu6wnrz.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.j8qai53wfm72vne.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.kj53a519kivhxro.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.kxqqjbmxp8ezmba.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.n91zwoayzk96u1e.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.nillb1q6fmhafgs.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.q0ljdrcqk1sztkw.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.syaa5bxtnosto28.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.v7a1ru04e6ptt8h.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.wygg97mtqdxwbm0.rcgu.o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19.1tczekdkosrwiv4i.rcgu.o" "-Wl,--as-needed" "-L" "/root/hello-request/target/debug/deps" "-L" "/root/hello-request/target/debug/build/openssl-sys-222ce05fd9e59354/out/openssl-build/install/lib" "-L" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/root/hello-request/target/debug/deps/libreqwest-32538d81c6e633ab.rlib" "/root/hello-request/target/debug/deps/libhyper_tls-fe48346797c26a2d.rlib" "/root/hello-request/target/debug/deps/libserde_urlencoded-4927a5f78976b92d.rlib" "/root/hello-request/target/debug/deps/libryu-0236f6dc03b61d91.rlib" "/root/hello-request/target/debug/deps/libbase64-ed528b1fb06aa435.rlib" "/root/hello-request/target/debug/deps/libipnet-13a6168997db870b.rlib" "/root/hello-request/target/debug/deps/libtokio_native_tls-8a0f2f75b1e25be5.rlib" "/root/hello-request/target/debug/deps/libmime-f9ab3ee664e16c5d.rlib" "/root/hello-request/target/debug/deps/libencoding_rs-c95afc8dee0b21c1.rlib" "/root/hello-request/target/debug/deps/libserde-7c8636b45e07af2d.rlib" "/root/hello-request/target/debug/deps/libnative_tls-8cc8058b5753906d.rlib" "/root/hello-request/target/debug/deps/libopenssl_probe-b0984027b9610f79.rlib" "/root/hello-request/target/debug/deps/libopenssl-c4b1afadb05077e2.rlib" "/root/hello-request/target/debug/deps/libbitflags-c4bc8217b5525fb8.rlib" "/root/hello-request/target/debug/deps/libforeign_types-8cd6591c8e9a7a5b.rlib" "/root/hello-request/target/debug/deps/libforeign_types_shared-e81d867e1a01b7d4.rlib" "/root/hello-request/target/debug/deps/libopenssl_sys-38fc5f5f28ce24e9.rlib" "/root/hello-request/target/debug/deps/libhyper-f3fb0bdd45e27bd9.rlib" "/root/hello-request/target/debug/deps/libwant-cb6a72520f754f6b.rlib" "/root/hello-request/target/debug/deps/libtry_lock-c84e6947d79ddb12.rlib" "/root/hello-request/target/debug/deps/libhttparse-664b208826ae8ceb.rlib" "/root/hello-request/target/debug/deps/libh2-540bafaf562f19c8.rlib" "/root/hello-request/target/debug/deps/libindexmap-00dd529f44c5e369.rlib" "/root/hello-request/target/debug/deps/libhashbrown-7ab00e879d52a799.rlib" "/root/hello-request/target/debug/deps/libslab-cc0771cd99431cfa.rlib" "/root/hello-request/target/debug/deps/libtokio_util-c757fc8adab1cc80.rlib" "/root/hello-request/target/debug/deps/libfutures_sink-13575af431ab24dc.rlib" "/root/hello-request/target/debug/deps/libtower_service-429c48a3ca3948bb.rlib" "/root/hello-request/target/debug/deps/libtracing-95b390d3163d0a73.rlib" "/root/hello-request/target/debug/deps/libtracing_core-dc3657c8fca2f58a.rlib" "/root/hello-request/target/debug/deps/libonce_cell-4de932266a0e37dc.rlib" "/root/hello-request/target/debug/deps/libfutures_channel-7979a9bb387bb67b.rlib" "/root/hello-request/target/debug/deps/libfutures_util-7133fd55e8a2cf45.rlib" "/root/hello-request/target/debug/deps/libfutures_task-655a43c858cd84ca.rlib" "/root/hello-request/target/debug/deps/libpin_utils-5598520b42f13dde.rlib" "/root/hello-request/target/debug/deps/libtokio-2c7897da8f344e98.rlib" "/root/hello-request/target/debug/deps/libsignal_hook_registry-f884911b5ff0b32b.rlib" "/root/hello-request/target/debug/deps/libnum_cpus-2aedabd589fb1a0a.rlib" "/root/hello-request/target/debug/deps/libsocket2-21d967793493e0a5.rlib" "/root/hello-request/target/debug/deps/libmio-816014f34292cdf6.rlib" "/root/hello-request/target/debug/deps/libparking_lot-3c2dea5a5410e2e6.rlib" "/root/hello-request/target/debug/deps/libparking_lot_core-3d9a6a25dab497a0.rlib" "/root/hello-request/target/debug/deps/liblibc-bd10417fc725967e.rlib" "/root/hello-request/target/debug/deps/libsmallvec-f27b5fed18292e6f.rlib" "/root/hello-request/target/debug/deps/liblock_api-75925c4e8221d0f3.rlib" "/root/hello-request/target/debug/deps/libscopeguard-f6a8af27eb217c03.rlib" "/root/hello-request/target/debug/deps/libhttp_body-e28a7545b7df9ab6.rlib" "/root/hello-request/target/debug/deps/libpin_project_lite-e8d359e0d6dc7857.rlib" "/root/hello-request/target/debug/deps/liblog-7c6ea07cd5f3dd79.rlib" "/root/hello-request/target/debug/deps/libcfg_if-0a7b5993b9fb90c4.rlib" "/root/hello-request/target/debug/deps/libfutures_core-27cadb881cc5ea31.rlib" "/root/hello-request/target/debug/deps/liburl-2c77c34df772237a.rlib" "/root/hello-request/target/debug/deps/libidna-4e617938e57720d7.rlib" "/root/hello-request/target/debug/deps/libunicode_normalization-c670285d8ca90c8f.rlib" "/root/hello-request/target/debug/deps/libtinyvec-11e40fd4c984d6b6.rlib" "/root/hello-request/target/debug/deps/libtinyvec_macros-46507314cc10f3b3.rlib" "/root/hello-request/target/debug/deps/libunicode_bidi-872d232d4bb95135.rlib" "/root/hello-request/target/debug/deps/libform_urlencoded-6dd790218656b268.rlib" "/root/hello-request/target/debug/deps/libpercent_encoding-a818f8d5ecd70566.rlib" "/root/hello-request/target/debug/deps/libhttp-3cc5745aa0e56d6e.rlib" "/root/hello-request/target/debug/deps/libitoa-cecc74b8f5d68e16.rlib" "/root/hello-request/target/debug/deps/libbytes-1828e3315b82be0f.rlib" "/root/hello-request/target/debug/deps/libfnv-81cf296291af575a.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd-8347dacf73ff8181.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libpanic_unwind-7c864a0072a3bfca.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libobject-f3140d2cdf33f2f8.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libmemchr-3e6e8d9adf95f603.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libaddr2line-c3214d789c863d6f.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libgimli-efc03ec9159535ab.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_demangle-925c57dfae9d0afd.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd_detect-0c087d6f5e18bb33.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libhashbrown-addbfbc0e8acd5a0.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libminiz_oxide-45a7ae677fa33eb5.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libadler-c798880eccacc08f.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_alloc-9f560ebd367f9b40.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libunwind-c8b557fe51fdf6ea.rlib" "-lunwind" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcfg_if-93959a4b4d0b2030.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-1c196568605b592d.rlib" "-lc" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liballoc-8e7bca5203171f71.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_core-cc2a13f3863fd692.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcore-dffeba2467220507.rlib" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-50d8ea67cf3a12a5.rlib" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-nostartfiles" "-L" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "-o" "/root/hello-request/target/debug/deps/hello_reqwest-1d244af24d214c19" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtend.o" "/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-50d8ea67cf3a12a5.rlib(cpu_model.o): in function `init_have_lse_atomics':
          /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.87/./lib/builtins/cpu_model.c:829: undefined reference to `getauxval'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `hello-reqwest` due to previous error

After inspecting the error message, I noticed the problem might on libcompiler_builtins-50d8ea67cf3a12a5.rlib, which came with rustup target add. Actually, they are part of LLVM's compiler-rt.

Steps

  1. Install Docker and start the container (docker run -ti --name alpine alpine:latest /bin/sh)
  2. Install the latest Rust (1.69.0)
  3. Set up a simple hello-reqwest project, with the content of main.rs and Cargo.toml mentioned above.
  4. Start the compilation.

Possible Solution(s)

No response

Notes

No response

Rustup version

$ rustup --version
rustup 1.26.0 (5af9b9484 2023-04-05)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.69.0 (84c898d65 2023-04-16)`

Installed toolchains

# rustup show
Default host: aarch64-unknown-linux-musl
rustup home:  /root/.rustup

stable-aarch64-unknown-linux-musl (default)
rustc 1.69.0 (84c898d65 2023-04-16)
@janvier-vip janvier-vip changed the title Got error undefined reference to getauxval’` during Rust cross-compile to aarch64 with musl on x86_64 Got error undefined reference to 'getauxval' during Rust cross-compile to aarch64 with musl on x86_64 Apr 18, 2023
@janvier-vip janvier-vip changed the title Got error undefined reference to 'getauxval' during Rust cross-compile to aarch64 with musl on x86_64 Got error undefined reference to 'getauxval' during the compilation on aarch64 with musl Apr 29, 2023
@funh2029
Copy link

When I set

[profile. release]
lto = true

I also encountered the same problem under musl when linking the main program.

error: linking with `aarch64-linux-musl-ld` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/root/.cargo/bin:/aarch64-linux-musl-cross/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "aarch64-linux-musl-ld" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtbegin.o" "/tmp/rustcmizhnM/symbols.o" "/srv/update/target/aarch64-unknown-linux-musl/release/deps/update-54860dc5c5b5817a.update.4921efe3-cgu.15.rcgu.o" "--as-needed" "-L" "/srv/update/target/aarch64-unknown-linux-musl/release/deps" "-L" "/srv/update/target/release/deps" "-L" "/srv/update/target/aarch64-unknown-linux-musl/release/build/openssl-sys-1d4012788120b9df/out/openssl-build/install/lib" "-L" "/srv/update/target/aarch64-unknown-linux-musl/release/build/bzip2-sys-c83a7b818eed5f5e/out/lib" "-L" "/srv/update/target/aarch64-unknown-linux-musl/release/build/zstd-sys-0f45ac252a1f6689/out" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Bstatic" "/tmp/rustcmizhnM/libopenssl_sys-3d2bb5be34f21e90.rlib" "/tmp/rustcmizhnM/libzstd_sys-bae0c08a6e77fa79.rlib" "/tmp/rustcmizhnM/libbzip2_sys-5f2c76fe50e60260.rlib" "-lunwind" "-lc" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-a546a2beee625b76.rlib" "-Bdynamic" "--eh-frame-hdr" "-z" "noexecstack" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "-o" "/srv/update/target/aarch64-unknown-linux-musl/release/deps/update-54860dc5c5b5817a" "--gc-sections" "-static" "-z" "relro" "-z" "now" "-O1" "--strip-all" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtend.o" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: aarch64-linux-musl-ld: /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-a546a2beee625b76.rlib(cpu_model.o): in function `init_have_lse_atomics':
          /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.87/./lib/builtins/cpu_model.c:1051: undefined reference to `getauxval'
          aarch64-linux-musl-ld: /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-a546a2beee625b76.rlib(cpu_model.o): in function `init_cpu_features':
          /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.87/./lib/builtins/cpu_model.c:1344: undefined reference to `getauxval'
          aarch64-linux-musl-ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.87/./lib/builtins/cpu_model.c:1345: undefined reference to `getauxval'

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

Remove "lto=true" and everything works fine.

@janvier-vip
Copy link
Author

But I don't have that option at all.

@Rustin170506
Copy link
Member

@janvier-vip Could you please provide a docker file or test repo then I can help test it and see what is going on. But I guess this is not a problem from Rustup maybe you should ask about it on Rust/Cargo repo.

@mitnk
Copy link

mitnk commented Aug 24, 2023

I also encountered this issue, and had a workaround with this:

$ uname -a
Linux FriendlyWrt 5.10.110 #1 SMP Tue Mar 14 21:59:07 CST 2023 aarch64 GNU/Linux

$ cat ~/.cargo/config.toml
[target.aarch64-unknown-linux-musl]
rustflags = ["-C", "link-arg=/root/.rustup/toolchains/stable-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/libc.a"]

This file libc.a is come with when installed rust with rustup.rs

@michalszmidt
Copy link

michalszmidt commented Aug 25, 2023

same issue, link to build log on CirrusCI.

  • rust from: rustup
  • rustc : 1.72.0
  • alpine: 3.18
  • platform: native aarch64

EDIT:

@mitnk
Your workaround doesn't work for me link to next build log

@michalszmidt
Copy link

Could anyone test if this happens for rust from vendor repo.
For instance on alpine?

@martadinata666
Copy link

I had this when using OpenSSL, moving to Rustls solved it. I don't know if that is also the culprit for other projects.

@michalszmidt
Copy link

michalszmidt commented Aug 25, 2023

Thanks @martadinata666 !
I also have openssl as a dependency. Needs investigating.

Anyone mention/crosspost this issue to official openssl and openssl rust crate repos?

@michalszmidt
Copy link

Ok, so I've setup a toy-repo with CirrusCI aarch64 runners on Alpine.

there are 3 configurations

  1. FAILS Explicitly selecting openssl: build log
  2. FAILS Bindings-over system tls: build log
  3. PASSES Rustls: build log

App does simple dns-over-tls query to example.com

alpine latest stable
rust latest stable from rustup

@michalszmidt
Copy link

michalszmidt commented Aug 26, 2023

I fetched rust from alpine repo and compilation passed.
This seems to be a workaround.
See those builds

Could you please test with other musl distros to detect where this bug occurs?
I mean, download rust from your distro repo and try to compile.

So what we know for now is:

  • Rust fetched from rustup on musl aarch64 will fail with openssl dependency
  • Rust fetched from rustup on musl aarch64 without openssl dependency will pass
  • Rust fetched from alpine with openssl dependency will pass
  • glibc and amd64 combinations will pass

@martadinata666
Copy link

martadinata666 commented Aug 26, 2023

I can confirm point no 1, 4. I dunno about number three, There are some patches from Alpine on their rust package https://git.alpinelinux.org/aports/tree/main/rust?h=master, maybe some key lay there.

@michalszmidt
Copy link

Duplicate?
rust-lang/rust#89626

CFLAGS=-mno-outline-atomics
appears to be solution in this thread, please someone test.

If it works for everyone, then this thread should be closed, and solution applied upstream sfackler/rust-openssl#2022

@michalszmidt
Copy link

michalszmidt commented Aug 26, 2023

I confirm that adding below to cargo.toml works for openssl and native-tls options
build logs

[target.aarch64-unknown-linux-musl]
rustflags = ["-C", "-mno-outline-atomics"]

somone else, please confirm.

@mitnk
Copy link

mitnk commented Aug 26, 2023

@michalszmidt Have you tried put the rustflags in file ~/.cargo/config.toml instead of Cargo.toml? I see there is warning unused manifest key: target.aarch64-unknown-linux-musl.rustflags in your build.

@michalszmidt
Copy link

@mitnk
I haven't noticed that one.
You mentioned old build with some other attempt to fix regression, but unfortunately it's also present in current build with above fix...

musl aarch64 build is waiting in queue.
If it fails, I'll try your solution

Thanks a lot!

@martadinata666
Copy link

What work for me is the linker change

docker@73c5942956cf:/app$ cat .cargo/config.toml 
[target.aarch64-unknown-linux-musl]
rustflags = ["-Clinker=rust-lld"]

@djc
Copy link
Contributor

djc commented Aug 28, 2023

So if I read the discussion correctly, there's not a rustup issue, just (potential issues) in Rust and/or rust-openssl?

@michalszmidt
Copy link

If it fails, I'll try your solution

Well, now I'm confused, as toy repo is fixed with rustflags -mno-outline-atomics but my actual app still fails with it and -Clinker=rust-lld by @martadinata666

  • first, fails with error: unknown codegen option: '-mno-outline-atomics'
  • second fails on openssl macros with
  = note: rust-lld: error: unable to find library -lgcc_s
          rust-lld: error: unable to find library -lc

build logs here

I'm a little bit tired with this issue, so I'll workaround this by fetching patched (slightly older) rust from alpine repo.

I'm still curious how other musl distros patch rust.
Could someone investigate?

@martadinata666
Copy link

martadinata666 commented Aug 31, 2023

I can build your actual app hctl with

[target.aarch64-unknown-linux-musl]
rustflags = ["-Clinker=rust-lld"]

I saw the run seems you placed the .cargo/config.toml in wrong place. Run log https://github.com/martadinata666/hctl/actions/runs/6037379261

@yogh333
Copy link

yogh333 commented Oct 24, 2023

Hello,

I have the same issue when cross-compiling for an ARM target from a Docker container based on Alpine:3.15:

= note: /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /opt/rustup/toolchains/nightly-2022-12-02-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-bc02f2db3c8aca29.rlib(cpu_model.o): undefined reference to symbol 'getauxval'
         /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /lib/libc.musl-aarch64.so.1: error adding symbols: DSO missing from command line
         collect2: error: ld returned 1 exit status

I have also noticed that getauxvalis only a weak symbol when dumping symbols from /lib/libc.musl-aarch64.so.1:

/usr/aarch64-alpine-linux-musl/bin/nm -AaD /lib/libc.musl-aarch64.so.1 | grep "getauxval"
/lib/libc.musl-aarch64.so.1:000000000003bf94 W getauxval

No trace of __getauxval as defined in musl.

@martadinata666
Copy link

the problem is somehow openssl called this issue "fixed" yet it still happening. So dunno where to start exactly.

@yogh333
Copy link

yogh333 commented Oct 25, 2023

I have no dependency with OpenSSL on my side. IMO it is linked with bindgen tool on Alpine when trying to to use it with 'static' feature enabled.

@Fishrock123
Copy link

I am encountering this with Alpine linux 3.18 in Docker targeting arm64.

cargo tree does not contain openssl or anything containing ssl for me.

A basic build with minimal / no dependencies does not have this error.


My issue appears to be related to the built crate with the git2 feature. It happens only when the git2 feature is enabled.

build.rs

fn main() {
    built::write_built_file().expect("Failed to acquire build-time information");
}

Cargo.toml:

[build-dependencies]
built = { version = "0.7.1", features = ["git2"] }

built's Cargo.toml has:

[dependencies]

git2 = { version = "0.18", optional = true, default-features = false, features = [] }

@Fishrock123
Copy link

Here is my linker error, presumably from libgit2-sys:

10.06 error: linking with `cc` failed: exit status: 1
10.06   |
10.06   = note: LC_ALL="C" PATH="/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/bin:/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/bin/self-contained:/usr/local/cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "cc" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtbegin.o" "/tmp/rustckg2trO/symbols.o" "/app/target/release/build/app-c0f4da874ae9cb79/build_script_build-c0f4da874ae9cb79.build_script_build.f21e8405a8680897-cgu.0.rcgu.o" "/app/target/release/build/app-c0f4da874ae9cb79/build_script_build-c0f4da874ae9cb79.5cs5zj006xi1kszr.rcgu.o" "-Wl,--as-needed" "-L" "/app/target/release/deps" "-L" "/app/target/release/build/libgit2-sys-b0265079e065ab06/out/build" "-L" "/app/target/release/build/libz-sys-c1e8829c1e9ea85f/out/lib" "-L" "/app/target/release/build/libz-sys-c1e8829c1e9ea85f/out/lib" "-L" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/app/target/release/deps/libbuilt-56c7d404c1513de9.rlib" "/app/target/release/deps/libgit2-6aa19713b6e193c1.rlib" "/app/target/release/deps/liburl-d29a55eea5f8985c.rlib" "/app/target/release/deps/libidna-1a73440288804038.rlib" "/app/target/release/deps/libunicode_normalization-76de1d961712a825.rlib" "/app/target/release/deps/libtinyvec-8d1e5b502fc4efc2.rlib" "/app/target/release/deps/libtinyvec_macros-069cbc3946ed02a8.rlib" "/app/target/release/deps/libunicode_bidi-cbc8e81f0702ea81.rlib" "/app/target/release/deps/libform_urlencoded-b5a5f80a2b2e2879.rlib" "/app/target/release/deps/libpercent_encoding-384b022fd3437573.rlib" "/app/target/release/deps/liblog-bbb4dc17ab6210af.rlib" "/app/target/release/deps/liblibgit2_sys-fedb06ffe4a81e54.rlib" "/app/target/release/deps/liblibz_sys-76293d2a5acbce7c.rlib" "/app/target/release/deps/liblibc-53797761d2e0bf05.rlib" "/app/target/release/deps/libbitflags-584c92aede70dcf2.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd-b13ff878e5d4772e.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libpanic_unwind-56ff67a6f2bbe26c.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libobject-b979ae03b63d8d68.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libmemchr-58cbe68a8bb2fe91.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libaddr2line-9e9fe0f60e5b4d08.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libgimli-d1dc1577e31bf044.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_demangle-121899391fa58bd9.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd_detect-e3f812d8bd3e0e87.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libhashbrown-8a773f116c633544.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_alloc-78fb926ae22d35c0.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libminiz_oxide-f64c502d42d93a67.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libadler-6e8fdf50da1f9c1f.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libunwind-594b86bee800abf8.rlib" "-lunwind" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcfg_if-f3811e9672bf18c2.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-30a29c0493545a48.rlib" "-lc" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liballoc-9c89ef503a72291b.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_core-4c4d965152b65e95.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcore-2b1e2f8589f4384b.rlib" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-330cf3f82d35bd41.rlib" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-nostartfiles" "-L" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "-o" "/app/target/release/build/app-c0f4da874ae9cb79/build_script_build-c0f4da874ae9cb79" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtend.o" "/usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
10.06   = note: /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-330cf3f82d35bd41.rlib(45c91108d938afe8-cpu_model.o): in function `init_have_lse_atomics':
10.06           /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.103/./lib/builtins/cpu_model.c:1075: undefined reference to `getauxval'
10.06           /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/local/rustup/toolchains/1.74.1-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-330cf3f82d35bd41.rlib(45c91108d938afe8-cpu_model.o): in function `init_cpu_features':
10.06           /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.103/./lib/builtins/cpu_model.c:1373: undefined reference to `getauxval'
10.06           /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.103/./lib/builtins/cpu_model.c:1374: undefined reference to `getauxval'
10.06           collect2: error: ld returned 1 exit status

@Fishrock123
Copy link

It seems the origin issue is rust-lang/rust#89626

@rami3l
Copy link
Member

rami3l commented Jul 10, 2024

I'm closing this as out of scope: this has nothing to do with the distribution of the Rust toolchain; in fact, this is not even a Rust-related issue as a the build failed when compiling a C library as a Rust project's dependency.

Please report the issue to the C library (or the *-sys crate) in question instead as we currently did in aws/aws-lc-rs#427 :)

@rami3l rami3l closed this as not planned Won't fix, can't repro, duplicate, stale Jul 10, 2024
@rami3l rami3l added the not-rustup Whatever is described in this issue isn't Rustup's fault label Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug not-rustup Whatever is described in this issue isn't Rustup's fault
Projects
None yet
Development

No branches or pull requests

10 participants