[ci] restrict CMake version for CUDA+Clang jobs #4104

StrikerRUS · 2021-03-24T13:12:45Z

Our CUDA+Clang builds are incopatible with yesterday-released CMake 3.20.0 version:

Found Clang, but using gcc.

-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/clang++ - skipped

...

gcc: error: unrecognized command line option ‘-fopenmp=libomp’; did you mean ‘-fopenmp-simd’?

-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The CUDA compiler identification is NVIDIA 10.0.130
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found OpenMP_C: -fopenmp=libomp (found version "3.1") 
-- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") 
-- Found OpenMP: TRUE (found version "3.1")  
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda (found suitable version "10.0", minimum required is "9.0") 
-- CMAKE_CUDA_FLAGS: -Xcompiler=-fopenmp=libomp -Xcompiler=-fPIC -Xcompiler=-Wall -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_62,code=sm_62 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -O3 -lineinfo
-- ALLFEATS_DEFINES: -DPOWER_FEATURE_WORKGROUPS=12;-DUSE_CONSTANT_BUF=0;-DENABLE_ALL_FEATURES
-- FULLDATA_DEFINES: -DPOWER_FEATURE_WORKGROUPS=12;-DUSE_CONSTANT_BUF=0;-DENABLE_ALL_FEATURES;-DIGNORE_INDICES
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Success
-- Using _mm_prefetch
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Success
-- Using _mm_malloc
-- Configuring done
CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256_sp".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256-fulldata_sp".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256_sp_const".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256-fulldata_sp_const".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "_lightgbm".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "_lightgbm".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "lightgbm".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "lightgbm".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256-allfeats_sp_const".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "histo_16_64_256-allfeats_sp".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /tmp/pip-req-build-83l4zq2o/build_cpp
[  2%] Building CUDA object CMakeFiles/histo_16_64_256_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o
[  5%] Building CUDA object CMakeFiles/histo_16_64_256-fulldata_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o
[  7%] Building CUDA object CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o
[ 10%] Building CUDA object CMakeFiles/histo_16_64_256_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o
gcc: error: unrecognized command line option ‘-fopenmp=libomp’; did you mean ‘-fopenmp-simd’?
CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/build.make:75: recipe for target 'CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o' failed
make[3]: *** [CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o] Error 1
make[3]: *** Deleting file 'CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o'
CMakeFiles/Makefile2:180: recipe for target 'CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/all' failed
make[2]: *** [CMakeFiles/histo_16_64_256-fulldata_sp_const.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
gcc: error: unrecognized command line option ‘-fopenmp=libomp’; did you mean ‘-fopenmp-simd’?
CMakeFiles/histo_16_64_256_sp_const.dir/build.make:75: recipe for target 'CMakeFiles/histo_16_64_256_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o' failed
make[3]: *** [CMakeFiles/histo_16_64_256_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o] Error 1
make[3]: *** Deleting file 'CMakeFiles/histo_16_64_256_sp_const.dir/src/treelearner/kernels/histogram_16_64_256.cu.o'
CMakeFiles/Makefile2:232: recipe for target 'CMakeFiles/histo_16_64_256_sp_const.dir/all' failed
make[2]: *** [CMakeFiles/histo_16_64_256_sp_const.dir/all] Error 2
gcc: error: unrecognized command line option ‘-fopenmp=libomp’; did you mean ‘-fopenmp-simd’?
CMakeFiles/histo_16_64_256-fulldata_sp.dir/build.make:75: recipe for target 'CMakeFiles/histo_16_64_256-fulldata_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o' failed
make[3]: *** [CMakeFiles/histo_16_64_256-fulldata_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o] Error 1
make[3]: *** Deleting file 'CMakeFiles/histo_16_64_256-fulldata_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o'
CMakeFiles/Makefile2:206: recipe for target 'CMakeFiles/histo_16_64_256-fulldata_sp.dir/all' failed
make[2]: *** [CMakeFiles/histo_16_64_256-fulldata_sp.dir/all] Error 2
gcc: error: unrecognized command line option ‘-fopenmp=libomp’; did you mean ‘-fopenmp-simd’?
CMakeFiles/histo_16_64_256_sp.dir/build.make:75: recipe for target 'CMakeFiles/histo_16_64_256_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o' failed
make[3]: *** [CMakeFiles/histo_16_64_256_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o] Error 1
make[3]: *** Deleting file 'CMakeFiles/histo_16_64_256_sp.dir/src/treelearner/kernels/histogram_16_64_256.cu.o'
CMakeFiles/Makefile2:125: recipe for target 'CMakeFiles/histo_16_64_256_sp.dir/all' failed
make[2]: *** [CMakeFiles/histo_16_64_256_sp.dir/all] Error 2
CMakeFiles/Makefile2:106: recipe for target 'CMakeFiles/_lightgbm.dir/rule' failed
make[1]: *** [CMakeFiles/_lightgbm.dir/rule] Error 2
Makefile:169: recipe for target '_lightgbm' failed
make: *** [_lightgbm] Error 2

StrikerRUS · 2021-03-24T14:03:06Z

.ci/setup.sh

@@ -98,7 +98,8 @@ else  # Linux
        apt-add-repository "deb https://apt.kitware.com/ubuntu/ $(lsb_release -cs) main" -y
        apt-get update
        apt-get install --no-install-recommends -y \
-            cmake
+            cmake="3.19.5-0kitware1" \
+            cmake-data="3.19.5-0kitware1"


Without cmake-data explicitly specified it yields the following:

Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: cmake : Depends: cmake-data (= 3.19.5-0kitware1) but 3.20.0-0kitware1 is to be installed E: Unable to correct problems, you have held broken packages.

StrikerRUS · 2021-03-24T15:18:07Z

@jameslamb Do you know any gods of CI we can pray to? 😃

jameslamb · 2021-03-24T15:20:38Z

@jameslamb Do you know any gods of CI we can pray to?

haha I've found many, none have ever answered my prayers.

jameslamb

oh interesting. Ok thanks for the investigation and quick fix!

This reverts commit ab474dc.

github-actions · 2023-08-23T23:57:41Z

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

restrict CMake version for CUDA jobs

a81ff7f

StrikerRUS added blocking maintenance labels Mar 24, 2021

StrikerRUS added 2 commits March 24, 2021 16:24

Update setup.sh

e0ad4a0

Update setup.sh

f10bdc9

StrikerRUS commented Mar 24, 2021

View reviewed changes

Update setup.sh

834fdda

StrikerRUS marked this pull request as ready for review March 24, 2021 15:18

StrikerRUS requested review from jameslamb and Laurae2 as code owners March 24, 2021 15:18

jameslamb approved these changes Mar 24, 2021

View reviewed changes

StrikerRUS changed the title ~~[ci] restrict CMake version for CUDA jobs~~ [ci] restrict CMake version for CUDA+Clang jobs Mar 24, 2021

StrikerRUS removed the blocking label Mar 24, 2021

StrikerRUS merged commit ab474dc into master Mar 24, 2021

StrikerRUS deleted the fix_ci branch March 24, 2021 15:35

jameslamb mentioned this pull request Mar 24, 2021

[dask] add support for eval sets and custom eval functions #4101

Merged

StrikerRUS mentioned this pull request Apr 3, 2021

[ci] upgrade R to 4.0.5 in CI #4158

Merged

StrikerRUS added a commit that referenced this pull request Apr 15, 2021

Revert "[ci] restrict CMake version for CUDA+Clang jobs (#4104)"

4a57243

This reverts commit ab474dc.

StrikerRUS mentioned this pull request Apr 15, 2021

[ci] unpin CMake version for CUDA + Clang toolchain #4183

Merged

github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] restrict CMake version for CUDA+Clang jobs #4104

[ci] restrict CMake version for CUDA+Clang jobs #4104

StrikerRUS commented Mar 24, 2021

StrikerRUS Mar 24, 2021

StrikerRUS commented Mar 24, 2021

jameslamb commented Mar 24, 2021

jameslamb left a comment

github-actions bot commented Aug 23, 2023

[ci] restrict CMake version for CUDA+Clang jobs #4104

[ci] restrict CMake version for CUDA+Clang jobs #4104

Conversation

StrikerRUS commented Mar 24, 2021

StrikerRUS Mar 24, 2021

Choose a reason for hiding this comment

StrikerRUS commented Mar 24, 2021

jameslamb commented Mar 24, 2021

jameslamb left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 23, 2023