Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression: undefined symbol __atomic_load_4 on risvc32imac-unknown-none-elf #85736

Open
xobs opened this issue May 27, 2021 · 10 comments
Open
Labels
A-atomic Area: Atomics, barriers, and sync primitives A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. ICEBreaker-Cleanup-Crew Helping to "clean up" bugs with minimal examples and bisections O-riscv Target: RISC-V architecture P-medium Medium priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another.

Comments

@xobs
Copy link
Contributor

xobs commented May 27, 2021

Description

When building for riscv32imac-unknown-none-elf, rustc is occasionally generating calls to __atomic_load_4. This symbol doesn't appear to exist, even though atomics are supported on this platform. Furthermore, I'm not sure if atomics are used anywhere in this codepath -- log is the only crate in the dependency tree that I haven't verified.

This works fine if I target riscv32i-unknown-none-elf, which disables atomic generation.

Furthermore, this builds fine if I use rustc 1.51, however it fails with rustc 1.52.1.

Background

I'm trying to work with the crate gdbstub to integrate it into a kernel running on baremetal RISC-V. The target is riscv32imac-unknown-none-elf, which has atomics. For some reason, the compiler is generating calls to __atomic_load_4, which doesn't exist.

Meta

rustc --version --verbose:

rustc 1.52.1 (9bc8c42bb 2021-05-09)
binary: rustc
commit-hash: 9bc8c42bb2f19e745a63f3445f1ac248fb015e53
commit-date: 2021-05-09
host: x86_64-unknown-linux-gnu
release: 1.52.1
LLVM version: 12.0.0
rustc 1.51

user@Cuboid:/mnt/d/Code/Xous/Core/kernel$ cargo +1.51 build --target riscv32imac-unknown-none-elf
warning: unused variable: `gdb_interrupt`
   --> src/debug.rs:168:13
    |
168 |             gdb_interrupt: GdbInterrupt<'_>,
    |             ^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_gdb_interrupt`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `register_thread`
   --> src/debug.rs:257:13
    |
257 |             register_thread: &mut dyn FnMut(Tid),
    |             ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_register_thread`

warning: field is never read: `pid`
   --> src/debug.rs:136:9
    |
136 |         pid: Option<xous_kernel::PID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: field is never read: `tid`
   --> src/debug.rs:137:9
    |
137 |         tid: Option<xous_kernel::TID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: field is never read: `thread_mask`
   --> src/debug.rs:138:9
    |
138 |         thread_mask: usize,
    |         ^^^^^^^^^^^^^^^^^^

warning: variant is never constructed: `Debug`
  --> src/services.rs:61:5
   |
61 |     Debug(usize),
   |     ^^^^^^^^^^^^

warning: 6 warnings emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.76s
user@Cuboid:/mnt/d/Code/Xous/Core/kernel$

rustc 1.52.1

user@Cuboid:/mnt/d/Code/Xous/Core/kernel$ cargo +stable build --target riscv32imac-unknown-none-elf
   Compiling kernel v0.8.2 (/mnt/d/Code/Xous/Core/kernel)
warning: unused variable: `gdb_interrupt`
   --> src/debug.rs:168:13
    |
168 |             gdb_interrupt: GdbInterrupt<'_>,
    |             ^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_gdb_interrupt`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `register_thread`
   --> src/debug.rs:257:13
    |
257 |             register_thread: &mut dyn FnMut(Tid),
    |             ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_register_thread`

warning: field is never read: `pid`
   --> src/debug.rs:136:9
    |
136 |         pid: Option<xous_kernel::PID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: field is never read: `tid`
   --> src/debug.rs:137:9
    |
137 |         tid: Option<xous_kernel::TID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: field is never read: `thread_mask`
   --> src/debug.rs:138:9
    |
138 |         thread_mask: usize,
    |         ^^^^^^^^^^^^^^^^^^

warning: variant is never constructed: `Debug`
  --> src/services.rs:61:5
   |
61 |     Debug(usize),
   |     ^^^^^^^^^^^^

error: linking with `rust-lld` failed: exit code: 1
  |
  = note: "rust-lld" "-flavor" "gnu" "-plugin-opt=O0" "-plugin-opt=mcpu=generic-rv32" "-L" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.0.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.1.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.10.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.11.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.12.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.13.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.14.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.15.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.2.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.3.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.4.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.5.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.6.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.7.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.8.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.9.rcgu.o" "-o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61" "--gc-sections" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps" "-L" "/mnt/d/Code/Xous/Core/kernel/target/debug/deps" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/kernel-b2cf221a795a62c2/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/kernel-b2cf221a795a62c2/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/riscv-4ccdf2c7a1922587/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/xous-1a7daf4c65ecd197/out" "-L" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib" "-Bstatic" "--whole-archive" "-lkernel" "--no-whole-archive" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libgdbstub_arch-56fd0be17381d6a4.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libxous-39428ed2b3a43c31.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libriscv-be84ff84a45e1b85.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbit_field-e83fefc77bfd4655.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbare_metal-4609200987315921.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libgdbstub-517875492397d3c0.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libcfg_if-5e47378e83c91cff.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libnum_traits-34dfc229d6370397.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libmanaged-9d0d80e606c7b727.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/liblog-4269a8ed40a0932a.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libcfg_if-2c07cfa8a1d4ab2f.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libutralib-18b768caf3ee14fc.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbitflags-46bd593bb6538427.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/librustc_std_workspace_core-525cd988d4e88ede.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/libcore-d1be523e2bc04c86.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/libcompiler_builtins-f400fa914b91ce44.rlib" "-Tlink.x" "-Bdynamic"
  = note: rust-lld: error: undefined symbol: __atomic_load_4
          >>> referenced by atomic.rs:0 (/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:0)
          >>>               lto.tmp:(core::sync::atomic::AtomicUsize::load::h0594e1bc6e9a0e87)
          >>> referenced by atomic.rs:0 (/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:0)
          >>>               lto.tmp:(core::sync::atomic::atomic_load::h507cc031d576ee38)


error: aborting due to previous error; 6 warnings emitted

error: could not compile `kernel`

To learn more, run the command again with --verbose.
user@Cuboid:/mnt/d/Code/Xous/Core/kernel$

@xobs xobs added the C-bug Category: This is a bug. label May 27, 2021
@xobs xobs changed the title rust-lld: error: undefined symbol: __atomic_load_4 on risvc32imac-unknown-none-elf Regression: undefined symbol __atomic_load_4 on risvc32imac-unknown-none-elf May 27, 2021
@jonas-schievink jonas-schievink added O-riscv Target: RISC-V architecture regression-from-stable-to-stable Performance or correctness regression from one stable version to another. labels May 27, 2021
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label May 27, 2021
@apiraino
Copy link
Contributor

@xobs if possible, could you provide a small repro of the regression? That would help in finding where the regression came from

@rustbot ping icebreakers-cleanup-crew

@rustbot rustbot added the ICEBreaker-Cleanup-Crew Helping to "clean up" bugs with minimal examples and bisections label May 27, 2021
@rustbot
Copy link
Collaborator

rustbot commented May 27, 2021

@xobs
Copy link
Contributor Author

xobs commented May 27, 2021

So this appears to be triggered by the "-C", "linker-plugin-lto=yes", flag. Has that been made obsolete?

I created a repro at https://github.com/xobs/rust-1.52.1-link-error but I couldn't get the repro any smaller. When I tried, the problem went away. This is demonstrated by using the bare-gdbstub crate instead of gdbstub.

My guess is that the function is getting large enough that it is... doing something. Even though the two calls are identical.

@SNCPlay42
Copy link
Contributor

Repro: build as a binary crate with -C linker-plugin-lto=yes (godbolt)

#![no_std]
#![no_main]

use core::sync::atomic::{Ordering, AtomicUsize};

#[panic_handler]
fn handle_panic(_arg: &core::panic::PanicInfo) -> ! {
    loop {}
}

#[no_mangle]
pub extern "C" fn _start() -> ! {
    let x = AtomicUsize::new(0);
    x.load(Ordering::SeqCst);
    loop { }
}

searched nightlies: from nightly-2021-02-05 to nightly-2021-05-04
regressed nightly: nightly-2021-03-18
searched commits: from f5d8117 to 36f1f04
regressed commit: 0c34122 (#83084)

bisected with cargo-bisect-rustc v0.6.0

Host triple: x86_64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --target=riscv32imac-unknown-none-elf --preserve --start=2021-02-05 --end=2021-05-04 -- rustc -- -C linker-plugin-lto=yes --target=riscv32imac-unknown-none-elf 

@apiraino
Copy link
Contributor

apiraino commented Jun 2, 2021

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-medium +I-nominated

@rustbot rustbot added P-medium Medium priority I-nominated and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Jun 2, 2021
@nagisa nagisa self-assigned this Jun 3, 2021
@nagisa nagisa added the A-LTO Area: Link-time optimization (LTO) label Jun 3, 2021
@nagisa nagisa removed their assignment Jul 8, 2021
@workingjubilee workingjubilee added the A-atomic Area: Atomics, barriers, and sync primitives label Aug 17, 2022
bors pushed a commit to rust-lang-ci/rust that referenced this issue Aug 5, 2023
This roughly reverts PR rust-lang#66548

Atomic "CAS" are still disabled for targets without the
*“A” Standard Extension for Atomic Instructions*.
However this extension only adds instructions for operations more complex
than simple loads and stores, which are always atomic when aligned.

In the [Unprivileged Spec v. 20191213](https://riscv.org/technical/specifications/)
section 2.6 *Load and Store Instructions* of
chapter 2 *RV32I Base Integer Instruction Set* (emphasis mine):

> Even when misaligned loads and stores complete successfully,
> these accesses might run extremely slowly depending on the implementation
> (e.g., when implemented via an invisible trap). Further-more, whereas
> **naturally aligned loads and stores are guaranteed to execute atomically**,
> misaligned loads and stores might not, and hence require
> additional synchronization to ensure atomicity.

Unfortunately PR rust-lang#66548 did not provide
much details on the bug that motivated it, but
rust-lang#66240 and
rust-lang#85736 appear related
and happen with targets that do have the A extension.
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 5, 2023
Re-enable atomic loads and stores for all RISC-V targets

This roughly reverts PR rust-lang#66548

Atomic "CAS" are still disabled for targets without the *“A” Standard Extension for Atomic Instructions*. However this extension only adds instructions for operations more complex than simple loads and stores, which are always atomic when aligned.

In the [Unprivileged Spec v. 20191213](https://riscv.org/technical/specifications/) section 2.6 *Load and Store Instructions* of chapter 2 *RV32I Base Integer Instruction Set* (emphasis mine):

> Even when misaligned loads and stores complete successfully, these accesses might run extremely slowly depending on the implementation (e.g., when implemented via an invisible trap). Further-more, whereas **naturally aligned loads and stores are guaranteed to execute atomically**, misaligned loads and stores might not, and hence require additional synchronization to ensure atomicity.

Unfortunately PR rust-lang#66548 did not provide much details on the bug that motivated it, but rust-lang#66240 and rust-lang#85736 appear related and happen with targets that do have the A extension.
@istankovic
Copy link
Contributor

Seems to be working with rustc 1.75.0-nightly (249624b 2023-10-20).

@istankovic
Copy link
Contributor

@xobs is this still a problem for you, with current rustc?

@xobs
Copy link
Contributor Author

xobs commented Jan 31, 2024

I haven't seen this problem in a while. Closing. Thank you for the fix!

@xobs xobs closed this as completed Jan 31, 2024
@xobs xobs reopened this Feb 1, 2024
@xobs
Copy link
Contributor Author

xobs commented Feb 1, 2024

We just had a user report an issue with Rust 1.75.0 today that was solved by changing LTO from "fat" to "thin", so this appears to still exist.

The error in question is:

  = note: rust-lld: error: undefined symbol: core::panicking::panic_nounwind::heb8ddfbf25c163db
          >>> referenced by compiler_builtins.3bf56cc9babbb60e-cgu.000
          >>>               compiler_builtins-caad9c73c64d0d39.compiler_builtins.3bf56cc9babbb60e-cgu.000.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h35d828d67d4905f6) in archive ~/.rustup/toolchains/1.75.0-aarch64-apple-darwin/lib/rustlib/riscv32imac-unknown-xous-elf/lib/libcompiler_builtins-caad9c73c64d0d39.rlib
          >>> referenced by compiler_builtins.3bf56cc9babbb60e-cgu.000
          >>>               compiler_builtins-caad9c73c64d0d39.compiler_builtins.3bf56cc9babbb60e-cgu.000.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h35d828d67d4905f6) in archive ~/.rustup/toolchains/1.75.0-aarch64-apple-darwin/lib/rustlib/riscv32imac-unknown-xous-elf/lib/libcompiler_builtins-caad9c73c64d0d39.rlib

Changing lto="thin results in a successful build.

@istankovic
Copy link
Contributor

This looks a bit different, this time it's not about __atomic_load_4 and seems to come from compiler_builtins.
Perhaps related to #112245?

@xobs could you please provide a minimal example that reproduces this issue? I.e. code and cargo and target configuration you're using, as well as the compiler invocation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-atomic Area: Atomics, barriers, and sync primitives A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. ICEBreaker-Cleanup-Crew Helping to "clean up" bugs with minimal examples and bisections O-riscv Target: RISC-V architecture P-medium Medium priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another.
Projects
None yet
Development

No branches or pull requests

8 participants