Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes for pkgsLLVM with llvmPackages_{15,16,git} #246577

Merged
merged 2 commits into from
Aug 22, 2023
Merged

Fixes for pkgsLLVM with llvmPackages_{15,16,git} #246577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6ba1b5b
llvmPackages_15, llvmPackages_16: Make the pkgsLLVM.stdenv work
pwaller Aug 1, 2023
044e8c4
llvmPackages_git: Port stdenv fix for cxx-headers
pwaller Aug 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions pkgs/development/compilers/llvm/15/libcxx/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, monorepoSrc, runCommand
, cmake, ninja, python3, fixDarwinDylibNames, version
, cxxabi ? if stdenv.hostPlatform.isFreeBSD then libcxxrt else libcxxabi
, libcxxabi, libcxxrt
, libcxxabi, libcxxrt, libunwind
, enableShared ? !stdenv.hostPlatform.isStatic

# If headersOnly is true, the resulting package would only include the headers.
Expand Down Expand Up @@ -62,7 +62,9 @@ stdenv.mkDerivation rec {
nativeBuildInputs = [ cmake ninja python3 ]
++ lib.optional stdenv.isDarwin fixDarwinDylibNames;

buildInputs = lib.optionals (!headersOnly) [ cxxabi ];
buildInputs =
lib.optionals (!headersOnly) [ cxxabi ]
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [ libunwind ];

cmakeFlags = let
# See: https://libcxx.llvm.org/BuildingLibcxx.html#cmdoption-arg-libcxx-cxx-abi-string
Expand All @@ -75,8 +77,18 @@ stdenv.mkDerivation rec {
"-DLIBCXX_CXX_ABI=${if headersOnly then "none" else libcxx_cxx_abi_opt}"
] ++ lib.optional (!headersOnly && cxxabi.libName == "c++abi") "-DLIBCXX_CXX_ABI_INCLUDE_PATHS=${cxxabi.dev}/include/c++/v1"
++ lib.optional (stdenv.hostPlatform.isMusl || stdenv.hostPlatform.isWasi) "-DLIBCXX_HAS_MUSL_LIBC=1"
++ lib.optional (stdenv.hostPlatform.useLLVM or false) "-DLIBCXX_USE_COMPILER_RT=ON"
++ lib.optionals stdenv.hostPlatform.isWasm [
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLIBCXX_USE_COMPILER_RT=ON"
# (Backport fix from 16, which has LIBCXX_ADDITIONAL_LIBRARIES, but 15
# does not appear to)
# There's precedent for this in llvm-project/libcxx/cmake/caches.
# In a monorepo build you might do the following in the libcxxabi build:
# -DLLVM_ENABLE_PROJECTS=libcxxabi;libunwind
# -DLIBCXXABI_STATICALLY_LINK_UNWINDER_IN_STATIC_LIBRARY=On
# libcxx appears to require unwind and doesn't pull it in via other means.
# "-DLIBCXX_ADDITIONAL_LIBRARIES=unwind"
"-DCMAKE_SHARED_LINKER_FLAGS=-lunwind"
] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXX_ENABLE_THREADS=OFF"
"-DLIBCXX_ENABLE_FILESYSTEM=OFF"
"-DLIBCXX_ENABLE_EXCEPTIONS=OFF"
Expand Down
7 changes: 7 additions & 0 deletions pkgs/development/compilers/llvm/15/libcxxabi/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ stdenv.mkDerivation rec {
] ++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLLVM_ENABLE_LIBCXX=ON"
"-DLIBCXXABI_USE_LLVM_UNWINDER=ON"
] ++ lib.optionals ((stdenv.hostPlatform.useLLVM or false) ||
(stdenv.hostPlatform.isDarwin && enableShared)) [
# libcxxabi's CMake looks as though it treats -nostdlib++ as implying -nostdlib,
# but that does not appear to be the case for example when building
# pkgsLLVM.libcxxabi (which uses clangNoCompilerRtWithLibc).
"-DCMAKE_EXE_LINKER_FLAGS=-nostdlib"
"-DCMAKE_SHARED_LINKER_FLAGS=-nostdlib"
Comment on lines +73 to +77
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for this? I would think we do wanna link libc but don't want to link (an already-existing) C++ standard library implementation. -nostdlib++ without -nostdlib therefore seems correct to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies, this is coming back to it some time after I implemented it, so I have forgotten the detail here and I have limited cycles to spare to dive back in. I suspect it has to do with making cmake configure, or linking succeed. I presume if you take it out you'll find the same failure I experienced? If not, maybe there is something further downstream which is broken.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK thanks @pwaller. So far, it actually seems to be working fine for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your comment is ambigous with two possible meanings:

  1. So far it works fine with the lines removed (as I suggested)
  2. So far it works fine as it currently is (with my lines present)

I read your intended response as (2).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pwaller: I think it's (1); see: c84a96b from #311836

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cross reference. I looked into it a little, I suspect the issue this was trying to fix went away when libcxxabi was merged into libcxx.

] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXXABI_ENABLE_THREADS=OFF"
"-DLIBCXXABI_ENABLE_EXCEPTIONS=OFF"
Expand Down
5 changes: 5 additions & 0 deletions pkgs/development/compilers/llvm/15/libunwind/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ stdenv.mkDerivation rec {
cd ../runtimes
'';

postInstall = lib.optionalString (enableShared && !stdenv.hostPlatform.isDarwin) ''
# libcxxabi wants to link to libunwind_shared.so (?).
ln -s $out/lib/libunwind.so $out/lib/libunwind_shared.so
'';

outputs = [ "out" "dev" ];

nativeBuildInputs = [ cmake ninja python3 ];
Expand Down
19 changes: 16 additions & 3 deletions pkgs/development/compilers/llvm/16/compiler-rt/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,16 @@ stdenv.mkDerivation {
++ lib.optional stdenv.isDarwin xcbuild.xcrun;
buildInputs = lib.optional stdenv.hostPlatform.isDarwin libcxxabi;

env.NIX_CFLAGS_COMPILE = toString [
env.NIX_CFLAGS_COMPILE = toString ([
"-DSCUDO_DEFAULT_OPTIONS=DeleteSizeMismatch=0:DeallocationTypeMismatch=0"
];
] ++ lib.optionals (!haveLibc) [
# The compiler got stricter about this, and there is a usellvm patch below
# which patches out the assert include causing an implicit definition of
# assert. It would be nicer to understand why compiler-rt thinks it should
# be able to #include <assert.h> in the first place; perhaps it's in the
# wrong, or perhaps there is a way to provide an assert.h.
"-Wno-error=implicit-function-declaration"
]);

cmakeFlags = [
"-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON"
Expand Down Expand Up @@ -102,7 +109,7 @@ stdenv.mkDerivation {
'' + lib.optionalString stdenv.isDarwin ''
substituteInPlace cmake/config-ix.cmake \
--replace 'set(COMPILER_RT_HAS_TSAN TRUE)' 'set(COMPILER_RT_HAS_TSAN FALSE)'
'' + lib.optionalString (useLLVM) ''
'' + lib.optionalString (useLLVM && !haveLibc) ''
substituteInPlace lib/builtins/int_util.c \
--replace "#include <stdlib.h>" ""
substituteInPlace lib/builtins/clear_cache.c \
Expand All @@ -117,6 +124,12 @@ stdenv.mkDerivation {
'' + lib.optionalString (useLLVM) ''
ln -s $out/lib/*/clang_rt.crtbegin-*.o $out/lib/crtbegin.o
ln -s $out/lib/*/clang_rt.crtend-*.o $out/lib/crtend.o
# Note the history of crt{begin,end}S in previous versions of llvm in nixpkg:
# The presence of crtbegin_shared has been added and removed; it's possible
# people have added/removed it to get it working on their platforms.
# Try each in turn for now.
ln -s $out/lib/*/clang_rt.crtbegin-*.o $out/lib/crtbeginS.o
ln -s $out/lib/*/clang_rt.crtend-*.o $out/lib/crtendS.o
ln -s $out/lib/*/clang_rt.crtbegin_shared-*.o $out/lib/crtbeginS.o
ln -s $out/lib/*/clang_rt.crtend_shared-*.o $out/lib/crtendS.o
'' + lib.optionalString doFakeLibgcc ''
Expand Down
8 changes: 8 additions & 0 deletions pkgs/development/compilers/llvm/16/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,14 @@ in let
[ "-rtlib=compiler-rt"
"-Wno-unused-command-line-argument"
"-B${targetLlvmLibraries.compiler-rt}/lib"

# Combat "__cxxabi_config.h not found". Maybe this could be fixed by
# copying these headers into libcxx? Note that building libcxx
# outside of monorepo isn't supported anymore, might be related to
# https://github.com/llvm/llvm-project/issues/55632
# ("16.0.3 libcxx, libcxxabi: circular build dependencies")
# Looks like the machinery changed in https://reviews.llvm.org/D120727.
"-I${lib.getDev targetLlvmLibraries.libcxx.cxxabi}/include/c++/v1"
]
++ lib.optional (!stdenv.targetPlatform.isWasm) "--unwindlib=libunwind"
++ lib.optional
Expand Down
19 changes: 13 additions & 6 deletions pkgs/development/compilers/llvm/16/libcxx/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, monorepoSrc, runCommand
, cmake, ninja, python3, fixDarwinDylibNames, version
, cxxabi ? if stdenv.hostPlatform.isFreeBSD then libcxxrt else libcxxabi
, libcxxabi, libcxxrt
, libcxxabi, libcxxrt, libunwind
, enableShared ? !stdenv.hostPlatform.isStatic

# If headersOnly is true, the resulting package would only include the headers.
Expand Down Expand Up @@ -47,8 +47,6 @@ stdenv.mkDerivation rec {

patches = [
./gnu-install-dirs.patch
] ++ lib.optionals stdenv.hostPlatform.isMusl [
../../libcxx-0001-musl-hacks.patch
];

postPatch = ''
Expand All @@ -62,7 +60,9 @@ stdenv.mkDerivation rec {
nativeBuildInputs = [ cmake ninja python3 ]
++ lib.optional stdenv.isDarwin fixDarwinDylibNames;

buildInputs = lib.optionals (!headersOnly) [ cxxabi ];
buildInputs =
lib.optionals (!headersOnly) [ cxxabi ]
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [ libunwind ];

cmakeFlags = let
# See: https://libcxx.llvm.org/BuildingLibcxx.html#cmdoption-arg-libcxx-cxx-abi-string
Expand All @@ -75,8 +75,15 @@ stdenv.mkDerivation rec {
"-DLIBCXX_CXX_ABI=${if headersOnly then "none" else libcxx_cxx_abi_opt}"
] ++ lib.optional (!headersOnly && cxxabi.libName == "c++abi") "-DLIBCXX_CXX_ABI_INCLUDE_PATHS=${cxxabi.dev}/include/c++/v1"
++ lib.optional (stdenv.hostPlatform.isMusl || stdenv.hostPlatform.isWasi) "-DLIBCXX_HAS_MUSL_LIBC=1"
++ lib.optional (stdenv.hostPlatform.useLLVM or false) "-DLIBCXX_USE_COMPILER_RT=ON"
++ lib.optionals stdenv.hostPlatform.isWasm [
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLIBCXX_USE_COMPILER_RT=ON"
# There's precedent for this in llvm-project/libcxx/cmake/caches.
# In a monorepo build you might do the following in the libcxxabi build:
# -DLLVM_ENABLE_PROJECTS=libcxxabi;libunwinder
# -DLIBCXXABI_STATICALLY_LINK_UNWINDER_IN_STATIC_LIBRARY=On
# libcxx appears to require unwind and doesn't pull it in via other means.
"-DLIBCXX_ADDITIONAL_LIBRARIES=unwind"
] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXX_ENABLE_THREADS=OFF"
"-DLIBCXX_ENABLE_FILESYSTEM=OFF"
"-DLIBCXX_ENABLE_EXCEPTIONS=OFF"
Expand Down
7 changes: 7 additions & 0 deletions pkgs/development/compilers/llvm/16/libcxxabi/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ stdenv.mkDerivation rec {
] ++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLLVM_ENABLE_LIBCXX=ON"
"-DLIBCXXABI_USE_LLVM_UNWINDER=ON"
] ++ lib.optionals ((stdenv.hostPlatform.useLLVM or false) ||
(stdenv.hostPlatform.isDarwin && enableShared)) [
# libcxxabi's CMake looks as though it treats -nostdlib++ as implying -nostdlib,
# but that does not appear to be the case for example when building
# pkgsLLVM.libcxxabi (which uses clangNoCompilerRtWithLibc).
"-DCMAKE_EXE_LINKER_FLAGS=-nostdlib"
"-DCMAKE_SHARED_LINKER_FLAGS=-nostdlib"
] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXXABI_ENABLE_THREADS=OFF"
"-DLIBCXXABI_ENABLE_EXCEPTIONS=OFF"
Expand Down
5 changes: 5 additions & 0 deletions pkgs/development/compilers/llvm/16/libunwind/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ stdenv.mkDerivation rec {
cd ../runtimes
'';

postInstall = lib.optionalString (enableShared && !stdenv.hostPlatform.isDarwin) ''
# libcxxabi wants to link to libunwind_shared.so (?).
ln -s $out/lib/libunwind.so $out/lib/libunwind_shared.so
'';

outputs = [ "out" "dev" ];

nativeBuildInputs = [ cmake ninja python3 ];
Expand Down
13 changes: 13 additions & 0 deletions pkgs/development/compilers/llvm/git/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,19 @@ in let
# what stdenv we use here, as long as CMake is happy.
cxx-headers = callPackage ./libcxx {
inherit llvm_meta;
# Note that if we use the regular stdenv here we'll get cycle errors
# when attempting to use this compiler in the stdenv.
#
# The final stdenv pulls `cxx-headers` from the package set where
# hostPlatform *is* the target platform which means that `stdenv` at
# that point attempts to use this toolchain.
#
# So, we use `stdenv_` (the stdenv containing `clang` from this package
# set, defined below) to sidestep this issue.
#
# Because we only use `cxx-headers` in `libcxxabi` (which depends on the
# clang stdenv _anyways_), this is okay.
stdenv = stdenv_;
headersOnly = true;
};

Expand Down
21 changes: 17 additions & 4 deletions pkgs/development/compilers/llvm/git/libcxx/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, monorepoSrc, runCommand
, cmake, ninja, python3, fixDarwinDylibNames, version
, cxxabi ? if stdenv.hostPlatform.isFreeBSD then libcxxrt else libcxxabi
, libcxxabi, libcxxrt
, libcxxabi, libcxxrt, libunwind
, enableShared ? !stdenv.hostPlatform.isStatic

# If headersOnly is true, the resulting package would only include the headers.
Expand Down Expand Up @@ -62,7 +62,10 @@ stdenv.mkDerivation rec {
nativeBuildInputs = [ cmake ninja python3 ]
++ lib.optional stdenv.isDarwin fixDarwinDylibNames;

buildInputs = lib.optionals (!headersOnly) [ cxxabi ];
buildInputs =
lib.optionals (!headersOnly) [ cxxabi ]
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [ libunwind ];


cmakeFlags = let
# See: https://libcxx.llvm.org/BuildingLibcxx.html#cmdoption-arg-libcxx-cxx-abi-string
Expand All @@ -75,8 +78,18 @@ stdenv.mkDerivation rec {
"-DLIBCXX_CXX_ABI=${if headersOnly then "none" else libcxx_cxx_abi_opt}"
] ++ lib.optional (!headersOnly && cxxabi.libName == "c++abi") "-DLIBCXX_CXX_ABI_INCLUDE_PATHS=${cxxabi.dev}/include/c++/v1"
++ lib.optional (stdenv.hostPlatform.isMusl || stdenv.hostPlatform.isWasi) "-DLIBCXX_HAS_MUSL_LIBC=1"
++ lib.optional (stdenv.hostPlatform.useLLVM or false) "-DLIBCXX_USE_COMPILER_RT=ON"
++ lib.optionals stdenv.hostPlatform.isWasm [
++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLIBCXX_USE_COMPILER_RT=ON"
# (Backport fix from 16, which has LIBCXX_ADDITIONAL_LIBRARIES, but 15
# does not appear to)
# There's precedent for this in llvm-project/libcxx/cmake/caches.
# In a monorepo build you might do the following in the libcxxabi build:
# -DLLVM_ENABLE_PROJECTS=libcxxabi;libunwinder
# -DLIBCXXABI_STATICALLY_LINK_UNWINDER_IN_STATIC_LIBRARY=On
# libcxx appears to require unwind and doesn't pull it in via other means.
# "-DLIBCXX_ADDITIONAL_LIBRARIES=unwind"
"-DCMAKE_SHARED_LINKER_FLAGS=-lunwind"
] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXX_ENABLE_THREADS=OFF"
"-DLIBCXX_ENABLE_FILESYSTEM=OFF"
"-DLIBCXX_ENABLE_EXCEPTIONS=OFF"
Expand Down
7 changes: 7 additions & 0 deletions pkgs/development/compilers/llvm/git/libcxxabi/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ stdenv.mkDerivation rec {
] ++ lib.optionals (stdenv.hostPlatform.useLLVM or false) [
"-DLLVM_ENABLE_LIBCXX=ON"
"-DLIBCXXABI_USE_LLVM_UNWINDER=ON"
] ++ lib.optionals ((stdenv.hostPlatform.useLLVM or false) ||
(stdenv.hostPlatform.isDarwin && enableShared)) [
# libcxxabi's CMake looks as though it treats -nostdlib++ as implying -nostdlib,
# but that does not appear to be the case for example when building
# pkgsLLVM.libcxxabi (which uses clangNoCompilerRtWithLibc).
"-DCMAKE_EXE_LINKER_FLAGS=-nostdlib"
"-DCMAKE_SHARED_LINKER_FLAGS=-nostdlib"
] ++ lib.optionals stdenv.hostPlatform.isWasm [
"-DLIBCXXABI_ENABLE_THREADS=OFF"
"-DLIBCXXABI_ENABLE_EXCEPTIONS=OFF"
Expand Down
5 changes: 5 additions & 0 deletions pkgs/development/compilers/llvm/git/libunwind/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ stdenv.mkDerivation rec {
cd ../runtimes
'';

postInstall = lib.optionalString (enableShared && !stdenv.hostPlatform.isDarwin) ''
# libcxxabi wants to link to libunwind_shared.so (?).
ln -s $out/lib/libunwind.so $out/lib/libunwind_shared.so
'';

outputs = [ "out" "dev" ];

nativeBuildInputs = [ cmake ninja python3 ];
Expand Down