[test suite] Running test-suite with non standard GCC's libatomic location

[Crivella]

I am building LLVM + all the included projects on top of GCC in a non standard location.

The problem I am having is that in the execution of the test-suite, several tests related to libunwind and libc++ link to libatomic, but during the execution of the compiled test with run.py the environment is not carried over.

When running the test-suite on my WS where a version of libatomic is present also in a standard location this is not a problem (guess it could still be if the version i am compiling of top of is incompatible with the one i have by default on my system?).
If the system does not have libatomic in a standard location (eg the build node I am using for testing), the test-suite gives many false failures of the type

********************
FAIL: llvm-libc++abi-shared.cfg.in :: uncaught_exception.pass.cpp (97463 of 97548)
******************** TEST 'llvm-libc++abi-shared.cfg.in :: uncaught_exception.pass.cpp' FAILED ********************
Exit Code: 127

Command Output (stdout):
--
# COMPILED WITH
~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/bin/clang++ ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/test/uncaught_exception.pass.cpp  --target=x86_64-unknown-linux-gnu -nostdinc++ -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/include -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/include/c++/v1 -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/include/x86_64-unknown-linux-gnu/c++/v1 -I "~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libunwind/include" -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/../libcxx/test/support -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/../libcxx/src -D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS -std=c++26 -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-noexcept-type -Wno-atomic-alignment -Wno-reserved-module-identifier -Wdeprecated-copy -Wdeprecated-copy-dtor -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -Wno-local-type-template-args -Wno-c++11-extensions -Wno-unknown-pragmas -Wno-pass-failed -Wno-mismatched-new-delete -Wno-redundant-move -Wno-self-move -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -Werror=thread-safety -Wuser-defined-warnings  -nostdlib++ -L ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/./lib/x86_64-unknown-linux-gnu -Wl,-rpath,~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/./lib/x86_64-unknown-linux-gnu -lc++ -lc++abi -pthread -latomic -o ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir/t.tmp.exe
# executed command: ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/bin/clang++ ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/test/uncaught_exception.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/include -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/include/c++/v1 -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/include/x86_64-unknown-linux-gnu/c++/v1 -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libunwind/include -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/../libcxx/test/support -I ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxxabi/../libcxx/src -D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS -std=c++26 -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-noexcept-type -Wno-atomic-alignment -Wno-reserved-module-identifier -Wdeprecated-copy -Wdeprecated-copy-dtor -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -Wno-local-type-template-args -Wno-c++11-extensions -Wno-unknown-pragmas -Wno-pass-failed -Wno-mismatched-new-delete -Wno-redundant-move -Wno-self-move -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -Werror=thread-safety -Wuser-defined-warnings -nostdlib++ -L ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/./lib/x86_64-unknown-linux-gnu -Wl,-rpath,~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/./lib/x86_64-unknown-linux-gnu -lc++ -lc++abi -pthread -latomic -o ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir/t.tmp.exe
# EXECUTED AS
~/Rocky8/haswell/software/Python/3.12.3-GCCcore-13.3.0/bin/python3.12 ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxx/utils/run.py --execdir ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir --  ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir/t.tmp.exe
# executed command: ~/Rocky8/haswell/software/Python/3.12.3-GCCcore-13.3.0/bin/python3.12 ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm-project-18.1.8.src/libcxx/utils/run.py --execdir ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir -- ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir/t.tmp.exe
# .---command stderr------------
# | ~/.local/easybuild/build/LLVM/18.1.8/GCCcore-13.3.0/llvm.obj.3/runtimes/runtimes-bins/test/Output/uncaught_exception.pass.cpp.dir/t.tmp.exe: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory
# `-----------------------------
# error: command failed with exit status: 127

I've looked a little to see if there is some option to pass to LIT_ARGS to bypass this by allowing all or some environment variable to be carried over to the test execution, but I haven't found anything.
The only other solution would be to hack the run.py or the configuration file before running the test.

I was wondering while probably niche, if this should be supported by the test-suite (or if it already is and I've missed it)

[Crivella]

As a bit of extra info this is the number of tests failing due to this error in the various configs of the test suite (the second number is the number of failures_due_to_libatomic + successes)

llvm-libc++-shared.cfg.in:        6880 / 8991
llvm-libc++abi-shared.cfg.in:       58 / 58
llvm-libunwind-shared.cfg.in:       11 / 12

[ldionne]

Since it's very difficult to support arbitrary use cases, the test suite was designed to be easily extensible and configurable by creating your own "base" config. You can define your own configuration file like libcxx/test/configs/llvm-libc++-shared.cfg.in and provide the compiler + linker flags of your choice in it. You can then point the test suite to that config file by using LIBCXX_TEST_CONFIG (& similar for libc++abi/libunwind) at CMake configuration time.

Please reopen if that approach doesn't solve your problem!