Support in building local Tinker9 on HPC

[jacekkozuch]

Dear all,

I am trying to build a local version of Tinker9 on a HPC. I'm not an expert in compiling, so probably I am missing something trivial (probably also some important info below) and I would be grateful for any support.

Currently, I am struggling with the nvc++ compiler. I've build a local version of HPC_SDK from NVIDIA to get OpenACC, which didn't show any errors. However, during the cmake I get the following issue:

-- Check for working CXX compiler: /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++ -- broken
CMake Error at /trinity/shared/easybuild/software/CMake/3.16.4-GCCcore-9.3.0/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake:53 (message):
The C++ compiler

"/home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++"

is not able to compile a simple test program.

I am using the following with gcc 9.3.0:
export CUDAHOME=/trinity/shared/easybuild/software/CUDAcore/11.0.2
export CUDACXX=$CUDAHOME/bin/nvcc
export FC=gfortran
export CXX=g++
export opt=release
export host=0
export prec=m
export compute_capability=60
export ACC=/home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++

The entire output from cmake is pasted below. Seems like issues with the HPC_SDK or compatibilities (?). Are there other sources to obtain OpenACC, if HPK_SDK is the issue here?

Big thanks in advance!

Best,
Jacek

-- The CXX compiler identification is PGI 21.9.0
-- Check for working CXX compiler: /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++
-- Check for working CXX compiler: /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++ -- broken
CMake Error at /trinity/shared/easybuild/software/CMake/3.16.4-GCCcore-9.3.0/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake:53 (message):
The C++ compiler

"/home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: /home/jkozuch/tinker9/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/gmake cmTC_1f34e/fast && /usr/bin/gmake -f CMakeFiles/cmTC_1f34e.dir/b

uild.make CMakeFiles/cmTC_1f34e.dir/build
gmake[1]: Entering directory /home/jkozuch/tinker9/CMakeFiles/CMakeTmp' Building CXX object CMakeFiles/cmTC_1f34e.dir/testCXXCompiler.cxx.o /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++ -o CMakeFiles/cmTC_1f34e. dir/testCXXCompiler.cxx.o -c /home/jkozuch/tinker9/CMakeFiles/CMakeTmp/testCXXCompiler.cxx Linking CXX executable cmTC_1f34e /trinity/shared/easybuild/software/CMake/3.16.4-GCCcore-9.3.0/bin/cmake -E cmake_link_script CMakeFi les/cmTC_1f34e.dir/link.txt --verbose=1 /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++ CMakeFiles/cmTC_1f34e.d ir/testCXXCompiler.cxx.o -o cmTC_1f34e /trinity/shared/easybuild/software/binutils/2.34-GCCcore-9.3.0/bin/ld: cannot find -lstdc++ gmake[1]: *** [cmTC_1f34e] Error 2 gmake[1]: Leaving directory /home/jkozuch/tinker9/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_1f34e/fast] Error 2

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:387 (enable_language)

-- Configuring incomplete, errors occurred!

[zhi-wang]

I just fixed an issue if CUDACXX is set, although I don't think it is relevant to the problem you saw here. First I'd like to edit these two lines

export CUDAHOME=/trinity/shared/easybuild/software/CUDAcore/11.0.2
export CUDACXX=$CUDAHOME/bin/nvcc

to

# assuming include, bin, etc. are under $cuda_dir
export cuda_dir=/trinity/shared/easybuild/software/CUDAcore/11.0.2

Remember to delete the CMake-generated files before re-running CMake again.

Second, if a CMakeLists.txt (Edit: CMakeCache.txt) is generated in your build directory and nvc++ etc. are already in your PATH, you can run
$tinker9/cmake/check-pgi inside your build directory. This is a script I wrote to check the possible compatibility issues. It won't fix your issue if there is any, but it'll help us understand the problem.

If none of the above works, please locate the file localrc and post its content as well as your g++ information here.

~/tinker9/build-3$ cat $(dirname $(which nvc++))/localrc
set LFC=-lgfortran;
set LDSO=/lib64/ld-linux-x86-64.so.2;
set GCCDIR=/usr/lib/gcc/x86_64-linux-gnu/9;
set G77DIR=/usr/lib/gcc/x86_64-linux-gnu/9/;
set OEM_INFO=64-bit target on x86-64 Linux $INFOTPVAL;
set GNUATOMIC=-latomic;
set GCCINC= /usr/lib/gcc/x86_64-linux-gnu/9/include /usr/local/include /usr/include/x86_64-linux-gnu /usr/include;
set GPPDIR= /usr/include/c++/9 /usr/include/x86_64-linux-gnu/c++/9 /usr/include/c++/9/backward /usr/lib/gcc/x86_64-linux-gnu/9/include /usr/local/include /usr/include/x86_64-linux-gnu /usr/include;
set LOCALRC=YES;
set EXTENSION=__extension__=;
set LC=$if(-Bstatic,-lgcc -lgcc_eh -lc -lgcc -lgcc_eh -lc, -lgcc -lc -lgcc -lgcc_s);
set DEFSTDOBJDIR=/usr/lib/x86_64-linux-gnu;
set DEFCUDAVERSION=11.4;
set DEFSTDPARCOMPUTECAP=75;
# GLIBC version 2.31
# GCC version 9.3.0
set GCCVERSION=90300;
export PGI=$COMPBASE;
# makelocalrc executed by root Mon 02 Aug 2021
$ g++ --version
g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[jacekkozuch]

I went through the changes you suggested.

• I changed the export ..., but still the same issue with the nvc++.

• When trying the check-pgi I obtain:
  "No CMakeCache.txt was found."

However, there is a CMakeCache.txt in my build folder.

• That's the result of my localrc:
  [jkozuch@g006 build]$ cat $(dirname $(which nvc++))/localrc
  set LFC=-lgfortran;
  set LDSO=/lib64/ld-linux-x86-64.so.2;
  set GCCDIR=/usr/lib/gcc/x86_64-redhat-linux/4.8.5;
  set G77DIR=/usr/lib/gcc/x86_64-redhat-linux/4.8.5/;
  set OEM_INFO=64-bit target on x86-64 Linux $INFOTPVAL;
  set GNUATOMIC=-latomic;
  set GCCINC= /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include /usr/local/include /u
  sr/include;
  set GPPDIR= /usr/include/c++/4.8.5 /usr/include/c++/4.8.5/x86_64-redhat-linux /u
  sr/include/c++/4.8.5/backward /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include /us
  r/local/include /usr/include;
  set NUMALIBNAME=-lnuma ;
  set LOCALRC=YES;
  set EXTENSION=extension=;
  set LC=$if(-Bstatic,-lgcc -lgcc_eh -lc -lgcc -lgcc_eh -lc, -lgcc -lc -lgcc -lgcc
  _s);
  set DEFCUDAVERSION=11.4;
  set DEFSTDPARCOMPUTECAP=;

GLIBC version 2.17

GCC version 4.8.5

set GCCVERSION=40805;
set LIBNCURSES=YES;
export PGI=$COMPBASE;

makelocalrc executed by jkozuch Thu Nov 4 17:39:22

Thanks a lot!

[jacekkozuch]

Sorry, forgot to copy check-pgi into the relevant folder. That's the output:

./check-pgi: line 28: --version: command not found ACC Compiler: /home/jkozuch/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++
ACC is configured to: Version 4.8.5 (of the Fortran compiler)
Fortran Compiler: Version

[zhi-wang]

You did not post your GCC version here but I guess it is 9.3.0 from your original post. I think this FAQ explained the problem and gave some solutions.

#140 (comment)

[jacekkozuch]

The nvc++ problem is solved by generating a localrc.
Thx so much for the help and sorry that I missed that in the FAQ.