NVTX Error During Build

[nap23carc]

After configuring the install, I'm getting an error about NVTX versions during the build stage. Any help would be greatly appreciated.

Operating System: Ubuntu 22.04.4 LTS
Kernel: Linux 6.8.0-40-generic
Architecture: x86-64

$ c++ --version
c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Aug_14_10:10:22_PDT_2024
Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0

Building CUDA source file fft.cu
In file included from /usr/local/cuda/include/cub/detail/nvtx3.hpp:635,
                 from /usr/local/cuda/include/cub/detail/nvtx.cuh:49,
                 from /usr/local/cuda/include/cub/device/device_for.cuh:40,
                 from /usr/local/cuda/include/thrust/system/cuda/detail/parallel_for.h:43,
                 from /usr/local/cuda/include/thrust/system/cuda/detail/swap_ranges.h:43,
                 from /usr/local/cuda/include/thrust/system/detail/adl/swap_ranges.h:50,
                 from /usr/local/cuda/include/thrust/detail/swap_ranges.inl:31,
                 from /usr/local/cuda/include/thrust/detail/swap.inl:30,
                 from /usr/local/cuda/include/thrust/swap.h:185,
                 from /usr/local/cuda/include/thrust/system/cuda/detail/iter_swap.h:34,
                 from /usr/local/cuda/include/thrust/system/detail/adl/iter_swap.h:50,
                 from /usr/local/cuda/include/thrust/detail/reference.h:37,
                 from /usr/local/cuda/include/thrust/memory.h:35,
                 from /usr/local/cuda/include/thrust/detail/allocator/temporary_allocator.h:31,
                 from /usr/local/cuda/include/thrust/detail/temporary_array.h:46,
                 from /usr/local/cuda/include/thrust/system/cuda/detail/internal/copy_cross_system.h:47,
                 from /usr/local/cuda/include/thrust/system/cuda/detail/copy.h:79,
                 from /usr/local/cuda/include/thrust/system/detail/adl/copy.h:50,
                 from /usr/local/cuda/include/thrust/detail/copy.inl:29,
                 from /usr/local/cuda/include/thrust/detail/copy.h:72,
                 from /usr/local/cuda/include/thrust/detail/allocator/copy_construct_range.inl:30,
                 from /usr/local/cuda/include/thrust/detail/allocator/copy_construct_range.h:45,
                 from /usr/local/cuda/include/thrust/detail/contiguous_storage.inl:29,
                 from /usr/local/cuda/include/thrust/detail/contiguous_storage.h:190,
                 from /usr/local/cuda/include/thrust/detail/vector_base.h:34,
                 from /usr/local/cuda/include/thrust/device_vector.h:33,
                 from fft.cu:46:
/usr/local/cuda/include/nvtx3/nvToolsExt.h:179:2: error: #error
   "Trying to #include NVTX version 3 in a source file where an older NVTX
   version has already been included.  If you are not directly using NVTX (the
   NVIDIA Tools Extension library), you are getting this error because
   libraries you are using have included different versions of NVTX.  Suggested
   solutions are: (1) reorder #includes so the newest NVTX version is included
   first, (2) avoid using the conflicting libraries in the same .c/.cpp file,
   or (3) update the library using the older NVTX version to use the newer
   version instead."
  179 | #error "Trying to #include NVTX version 3 in a source f
ile where an older NVTX version has already been included.  If you are not direc
tly using NVTX (the NVIDIA Tools Extension library), you are getting this error
because libraries you are using have included different versions of NVTX.  Sugge
sted solutions are: (1) reorder #includes so the newest NVTX version is included
first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3
) update the library using the older NVTX version to use the newer version inste
ad."
      |  ^~~~~
make[1]: *** [autodep.mk:52: fft.o] Error 1
make[1]: Leaving directory '/home/software/bifrost/src'
make: *** [Makefile:22: libbifrost] Error 2

[jaycedowell]

Looks like this is sneaking in from fft.cu including thrust/device_vector.h and trace.hpp. trace.hpp looks to be pulling in the older version of NVTX on line 35.

[jaycedowell]

How crazy is the traceback if you modify trace.hpp to change:

#if BF_CUDA_ENABLED
#include <nvToolsExt.h>
#endif

to

#if BF_CUDA_ENABLED
#ifndef NVTX_VERSION
#include <nvToolsExt.h>
#endif
#endif

?

[nap23carc]

No crazier. Same error in the build process.

[dentalfloss1]

I tested cuda toolkit and driver versions recently. Unfortunately 12.6 was not included in those tests ( I don't think it was out yet). Ideally, we would like to make sure 12.6 is working with Bifrost, however if you continue to run into roadblocks you might want to try the cuda 12.4 toolkit and driver. It seems to play nicely with Bifrost.

[nap23carc]

Thank you! I'll take 12.4 for a lap around the block.

[nap23carc]

Installing the Cuda Toolkit 12.4 allowed the build and install processes to complete. libbifrost.so and libbifrost.so.0 compiled successfully. Thanks for the help.