I've got the same error while compiling deepmd-kit #272

hanyecn · 2020-10-08T19:28:32Z

hanyecn
Oct 8, 2020

Hello, I met the same problem while compiling deepmd-kit.
I followed the instructions from tensorflow website and deepmd-kit github homepage.
gcc 7.5
cuda 10.1
cudnn 7.6
tensorrt 6
tensorflow 2.1.2
bazel 0.29.1
nccl 2.7.8
python3

I compiled pip package and c++ lib file of tensorflow-gpu at the same time:
bazel build --config=opt --config=mkl --copt=-msse4.2 --copt=-mavx2 --action_env=PATH --action_env=LD_LIBRARY_PATH
//tensorflow/tools/pip_package //tensorflow:libtensorflow_cc.so

the pip_package was installed successfully and works well, I can do 'import tensorflow as tf; tf.print()'
but when I tried to compile deepmd-kit using
cmake -DUSE_CUDA_TOOLKIT=true --DXDRFILE_ROOT=$xdrfile_root --DTENSORFLOW_ROOT=$tensorflow_root

I got the same error:

Not found 'tensorflow/core/public/session.h' directory in path

I am sure the 'session.h' file IS there in the directory.

Please help me.

Thanks a lot.

njzjz · 2020-10-09T09:15:40Z

njzjz
Oct 9, 2020
Maintainer

Do you set $tensorflow_root? Can you ensure $tensorflow_root/include/tensorflow/core/public/session.h exisits?

0 replies

hanyecn · 2020-10-09T09:49:44Z

hanyecn
Oct 9, 2020
Author

Do you set $tensorflow_root? Can you ensure $tensorflow_root/include/tensorflow/core/public/session.h exisits?

actually I didn't export enverimental variable of $tensorflow_root, but when I tried to compile with cmake, I used the flag of -DTENSORFLOW_ROOT which links the tensorflow root dir. And I am sure the file of session is in the correct position. thanks for you reply and looking for your further instructions.

0 replies

njzjz · 2020-10-09T10:07:11Z

njzjz
Oct 9, 2020
Maintainer

Which one did you use, -DTENSORFLOW_ROOT or --DTENSORFLOW_ROOT? I mean how many - did you input?

0 replies

hanyecn · 2020-10-09T10:09:12Z

hanyecn
Oct 9, 2020
Author

-DTENSORFLOW_ROOT, should I use --DTENSORFLOW_ROOT=......?

0 replies

hanyecn · 2020-10-09T10:17:54Z

hanyecn
Oct 9, 2020
Author

[hanye@hanye ~]$ source venv/bin/activate
(venv) [hanye@hanye ~]$ export CC=which gcc
(venv) [hanye@hanye ~]$ export CXX=which g++
(venv) [hanye@hanye ~]$ cd Downloads/deepmd-kit
(venv) [hanye@hanye deepmd-kit]$ cd source/build/
(venv) [hanye@hanye build]$ rm -rf *
(venv) [hanye@hanye build]$ cmake -DUSE_CUDA_TOOLKIT=true -DXDRFILE_ROOT=/home/hanye/xdrfile -DTENSORFLOW_ROOT=/home/hanye/tensorflow_2.1_gpu -DCMAKE_INSTALL_PREFIX=/home/hanye/deepmd_kit ../
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features

-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "1.8.3.1")
CMake Error at cmake/Findtensorflow.cmake:45 (message):
Not found 'tensorflow/core/public/session.h' directory in path
'/home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/usr/;/usr/local/'
You can manually set the tensorflow install path by -DTENSORFLOW_ROOT
Call Stack (most recent call first):
CMakeLists.txt:50 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/hanye/Downloads/deepmd-kit/source/build/CMakeFiles/CMakeOutput.log".
(venv) [hanye@hanye build]$
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h

0 replies

njzjz · 2020-10-09T10:23:33Z

njzjz
Oct 9, 2020
Maintainer

Oh got it. Please check whether these files all exist:

deepmd-kit/source/cmake/Findtensorflow.cmake

Lines 26 to 30 in 59d780c

    
           tensorflow/core/public/session.h 
        
           tensorflow/core/platform/env.h 
        
           tensorflow/core/framework/op.h 
        
           tensorflow/core/framework/op_kernel.h 
        
           tensorflow/core/framework/shape_inference.h

0 replies

hanyecn · 2020-10-09T10:27:25Z

hanyecn
Oct 9, 2020
Author

All these files are there.
(venv) [hanye@hanye build]$
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/platform/env.
env.cc env.h
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/platform/env.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/platform/env.h
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/op.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/op.h
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/op_kernel.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/op_kernel.h
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/shape_inference.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/framework/shape_inference.h

0 replies

hanyecn · 2020-10-09T13:01:14Z

hanyecn
Oct 9, 2020
Author

Any further suggestions? Thanks.

0 replies

hanyecn · 2020-10-09T15:50:38Z

hanyecn
Oct 9, 2020
Author

if (NOT TensorFlow_INCLUDE_DIRS AND tensorflow_FIND_REQUIRED)
message(FATAL_ERROR
"Not found 'tensorflow/core/public/session.h' directory in path '${TensorFlow_search_PATHS}' "
"You can manually set the tensorflow install path by -DTENSORFLOW_ROOT ")
endif ()

What's the meaning of "tensorflow_FIND_REQUIRED"? I am almost getting out of my mind......

0 replies

njzjz · 2020-10-09T16:23:47Z

njzjz
Oct 9, 2020
Maintainer

What's your cmake version? If it is larger than or equal to 3.17, pass either --debug-find or -DCMAKE_FIND_DEBUG_MODE=true to see what happened here.

0 replies

hanyecn · 2020-10-09T16:30:15Z

hanyecn
Oct 9, 2020
Author

my cmake version is 3.11.0,
and I got the same error after passed --debug-find or -DCMAKE_FIND_DEBUG_MODE=true.

0 replies

hanyecn · 2020-10-11T15:53:40Z

hanyecn
Oct 11, 2020
Author

I've tried to do some debug works by inserting following lines into the Findtensorflow.cmake:

includes

find_path(TensorFlow_INCLUDE_DIRS
NAMES
tensorflow/core/public/session.h
tensorflow/core/platform/env.h
tensorflow/core/framework/op.h
tensorflow/core/framework/op_kernel.h
tensorflow/core/framework/shape_inference.h
PATHS ${TensorFlow_search_PATHS}
PATH_SUFFIXES "/include"
NO_DEFAULT_PATH
)
message(STATUS "session.h: ${TensorFlow_INCLUDE_DIRS}")
message(STATUS "${TensorFlow_INCLUDE_DIRS}")
message(STATUS "${tensorflow_FIND_REQUIRED}")

then cmake, the output is:"
(venv) [hanye@hanye build]$ cmake -DUSE_CUDA_TOOLKIT=true -DXDRFILE_ROOT=/home/hanye/xdrfile -DTENSORFLOW_ROOT=/home/hanye/tensorflow_2.1_gpu -DCMAKE_INSTALL_PREFIX=/home/hanye/deepmd_kit ../
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features

-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "1.8.3.1")
-- session.h: /home/hanye/tensorflow_2.1_gpu/include
-- /home/hanye/tensorflow_2.1_gpu/include
--1
CMake Error at cmake/Findtensorflow.cmake:45 (message):
Not found 'tensorflow/core/public/session.h' directory in path
'/home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/usr/;/usr/local/'
You can manually set the tensorflow install path by -DTENSORFLOW_ROOT
Call Stack (most recent call first):
CMakeLists.txt:50 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/hanye/Downloads/deepmd-kit/source/build/CMakeFiles/CMakeOutput.log".
(venv) [hanye@hanye build]$
(venv) [hanye@hanye build]$ ls /home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h
/home/hanye/tensorflow_2.1_gpu/include/tensorflow/core/public/session.h
"
which means that find_path has found the "session.h" and stored the path to the variable "TensorFlow_INCLUDE_DIRS", to my understanding, is that correct?

0 replies

hanyecn · 2020-10-11T16:41:31Z

hanyecn
Oct 11, 2020
Author

I've found the reason of this problem, it is because that I didn't install protobuf. When I tried to compile again after I installed protobuf3.8.0, I got following informations:
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc
-- Check for working C compiler: /home/hanye/opt/gcc7.5/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++
-- Check for working CXX compiler: /home/hanye/opt/gcc7.5/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "1.8.3.1")
-- session.h: /home/hanye/tensorflow_2.1_gpu/include
-- /home/hanye/tensorflow_2.1_gpu/include
-- 1
-- /home/hanye/tensorflow_2.1_gpu/include;/usr/local/include
-- Enabled cpp interface build, looking for tensorflow_cc and tensorflow_framework
-- Found TensorFlow: /home/hanye/tensorflow_2.1_gpu/include;/usr/local/include, /home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_cc.so;/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so, /home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so in /home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/home/hanye/tensorflow_2.1_gpu;/home/hanye/tensorflow_2.1_gpu/../tensorflow_core;/usr/;/usr/local/
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
CMake Error at CMakeLists.txt:85 (message):
Failed to compile:

Change Dir: /home/hanye/Downloads/deepmd-kit/source/build/tf_cxx_abi/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/gmake" "cmTC_ee0cc/fast"

/usr/bin/gmake -f CMakeFiles/cmTC_ee0cc.dir/build.make
CMakeFiles/cmTC_ee0cc.dir/build

gmake[1]: Entering directory
`/home/hanye/Downloads/deepmd-kit/source/build/tf_cxx_abi/CMakeFiles/CMakeTmp'

Building CXX object CMakeFiles/cmTC_ee0cc.dir/tf_cxx_abi.cpp.o

/home/hanye/opt/gcc7.5/bin/g++ -I/home/hanye/tensorflow_2.1_gpu/include
-I/usr/local/include -std=c++11 -Wno-ignored-attributes -o
CMakeFiles/cmTC_ee0cc.dir/tf_cxx_abi.cpp.o -c
/home/hanye/Downloads/deepmd-kit/source/cmake/tf_cxx_abi.cpp

Linking CXX executable cmTC_ee0cc

/home/hanye/venv/lib/python3.6/site-packages/cmake/data/bin/cmake -E
cmake_link_script CMakeFiles/cmTC_ee0cc.dir/link.txt --verbose=1

/home/hanye/opt/gcc7.5/bin/g++ -std=c++11 -Wno-ignored-attributes
CMakeFiles/cmTC_ee0cc.dir/tf_cxx_abi.cpp.o -o cmTC_ee0cc
-Wl,-rpath,/home/hanye/tensorflow_2.1_gpu/lib
/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so

/usr/bin/ld: warning: libiomp5.so, needed by
/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so, not found
(try using -rpath or -rpath-link)

/usr/bin/ld: warning: libmklml_intel.so, needed by
/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so, not found
(try using -rpath or -rpath-link)

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_gemm_s8u8s32_pack'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `omp_set_num_threads@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sscal'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_saxpy'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sgemm_pack'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `omp_in_parallel@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `vmsExp'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_gemm_s8u8s32_pack_get_size'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_gemm_s8u8s32_compute'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sgemm'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `i_calloc'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `omp_get_max_threads@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `omp_get_thread_num@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `i_malloc'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sgemm_compute'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `GOMP_barrier@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `omp_get_num_threads@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `i_free'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sasum'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_gemm_s8u8s32'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `cblas_sgemm_pack_get_size'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `GOMP_parallel@VERSION'

/home/hanye/tensorflow_2.1_gpu/lib/libtensorflow_framework.so: undefined
reference to `i_realloc'

collect2: error: ld returned 1 exit status

gmake[1]: *** [cmTC_ee0cc] Error 1

gmake[1]: Leaving directory
`/home/hanye/Downloads/deepmd-kit/source/build/tf_cxx_abi/CMakeFiles/CMakeTmp'

gmake: *** [cmTC_ee0cc/fast] Error 2

-- Configuring incomplete, errors occurred!
See also "/home/hanye/Downloads/deepmd-kit/source/build/CMakeFiles/CMakeOutput.log".
See also "/home/hanye/Downloads/deepmd-kit/source/build/CMakeFiles/CMakeError.log".

Any suggestions?
Thanks.
I'll keep it updated.

0 replies

hanyecn · 2020-10-11T18:26:41Z

hanyecn
Oct 11, 2020
Author

according to the solution from https://fantashit.com/config-mkl-leads-to-libmklml-intel-so-cannot-open-shared-object-file-no-such-file-or-directory/, I compiled tensorflw/external/mkl_dnn and install the libraries. After that, compiling of deepmd-kit gives no errors.
but when I tried "dp -h" after install, I got these informations:
(venv) [hanye@hanye ~]$ dp -h
2020-10-11 14:20:22.677749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-10-11 14:20:22.680158: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
WARNING:tensorflow:From /home/hanye/venv/lib/python3.6/site-packages/tensorflow_core/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
File "/home/hanye/venv/bin/dp", line 5, in
from deepmd.main import main
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/init.py", line 1, in
from .env import set_mkl
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/env.py", line 61, in
op_module = get_module("libop_abi")
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/env.py", line 58, in get_module
module = tf.load_op_library(module_path + "{}.{}".format(module_name, ext))
File "/home/hanye/venv/lib/python3.6/site-packages/tensorflow_core/python/framework/load_library.py", line 57, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/hanye/venv/lib/python3.6/site-packages/deepmd/libop_abi.so: undefined symbol: _ZN10tensorflow8OpKernel11TraceStringEPNS_15OpKernelContextEb

any suggestions?
I'll keep this updated.
thanks a lot.

0 replies

njzjz · 2020-10-11T18:53:12Z

njzjz
Oct 11, 2020
Maintainer

according to the solution from https://fantashit.com/config-mkl-leads-to-libmklml-intel-so-cannot-open-shared-object-file-no-such-file-or-directory/, I compiled tensorflw/external/mkl_dnn and install the libraries. After that, compiling of deepmd-kit gives no errors.
but when I tried "dp -h" after install, I got these informations:
(venv) [hanye@hanye ~]$ dp -h
2020-10-11 14:20:22.677749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-10-11 14:20:22.680158: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
WARNING:tensorflow:From /home/hanye/venv/lib/python3.6/site-packages/tensorflow_core/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
File "/home/hanye/venv/bin/dp", line 5, in
from deepmd.main import main
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/init.py", line 1, in
from .env import set_mkl
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/env.py", line 61, in
op_module = get_module("libop_abi")
File "/home/hanye/venv/lib/python3.6/site-packages/deepmd/env.py", line 58, in get_module
module = tf.load_op_library(module_path + "{}.{}".format(module_name, ext))
File "/home/hanye/venv/lib/python3.6/site-packages/tensorflow_core/python/framework/load_library.py", line 57, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/hanye/venv/lib/python3.6/site-packages/deepmd/libop_abi.so: undefined symbol: _ZN10tensorflow8OpKernel11TraceStringEPNS_15OpKernelContextEb

any suggestions?
I'll keep this updated.
thanks a lot.

dp is Python interface, which is independent with C++ interface. How do you install it?

0 replies

Mskater · 2020-10-13T22:52:09Z

Mskater
Oct 13, 2020

@hanyecn Did your procedure above fix all of the connections to the python library file libop_abi.so,
and the C++ connection for session.h ?
I did your procedure, and still have the same problems.
Anyone, I'm still stuck. Thanks.

0 replies

hanyecn · 2020-10-14T07:48:37Z

hanyecn
Oct 14, 2020
Author

@hanyecn Did your procedure above fix all of the connections to the python library file libop_abi.so,
and the C++ connection for session.h ?
I did your procedure, and still have the same problems.
Anyone, I'm still stuck. Thanks.

yes, it works for me. now the code can work correctly, good luck.

0 replies

njzjz · 2020-10-14T15:01:43Z

njzjz
Oct 14, 2020
Maintainer

But why only one of my GPUs is working?

This is expected behavior

0 replies

hanyecn · 2020-10-14T15:03:31Z

hanyecn
Oct 14, 2020
Author

But why only one of my GPUs is working?

This is expected behavior

Thanks for your help.

0 replies

Mskater · 2020-10-15T02:43:30Z

Mskater
Oct 15, 2020

@hanyecn Can you please post your exact cmake build command if that worked for you? I need a good example. Mine still doesn’t work after reinstalling deepmd-kit . Can’t find session.h...
Anyone,
Also I was wondering if it is possible to build it with bazel instead since cmake has this problem.? And how to do it?
Thanks for advice as always!

0 replies

hanyecn · 2020-10-15T09:44:55Z

hanyecn
Oct 15, 2020
Author

My cmake build command is: cmake -DUSE_CUDA_TOOLKIT=true -DXDRFILE_ROOT=/home/hanye/xdrfile -DTENSORFLOW_ROOT=/home/hanye/tensorflow_2.1_gpu -DCMAKE_INSTALL_PREFIX=/home/hanye/deepmd_kit ../
BUT, it won't work when I try 'dp -h', the same errors come to me. Anyway, I believe that if you are trying to install deepmd-kit using the *.sh file, you do not need to build the package of deepmd-kit from scratch yourself. Wish you good luck!

0 replies

hanyecn · 2020-10-18T17:09:36Z

hanyecn
Oct 18, 2020
Author

Hi,
When I tried to perform MD simulations of water which is the sample of deepmd-kit using the lmp installed, the following errors arose, any suggestions?

Info of model(s):
using 1 model(s): ../train/graph.pb
rcut in model: 6
ntypes in model: 2
Neighbor list info ...
update every 10 steps, delay 0 steps, check no
max neighbors/atom: 2000, page size: 100000
master list distance cutoff = 8
ghost atom cutoff = 8
binsize = 4, bins = 4 4 4
1 neighbor lists, perpetual/occasional/extra = 1 0 0
(1) pair deepmd, perpetual
attributes: , newton on
pair build: full/bin/atomonly
stencil: full/bin/3d
bin: standard
Setting up Verlet run ...
Unit style : metal
Current step : 0
Time step : 0.0005
2020-10-18 13:02:01.121037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
Segmentation fault (core dumped)

Thanks a lot!

0 replies

njzjz · 2020-10-18T19:56:47Z

njzjz
Oct 18, 2020
Maintainer

I think you met a similar error to #262 (comment). Can you list your environment variables?

0 replies

hanyecn · 2020-10-19T10:13:40Z

hanyecn
Oct 19, 2020
Author

I think you met a similar error to #262 (comment). Can you list your environment variables?

Thanks for you reply, my environment variables are shown below:

(venv) [hanye@hanye ~]$ env
XDG_SESSION_ID=117
HOSTNAME=hanye
SELINUX_ROLE_REQUESTED=
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=192.168.111.11 60735 22
SELINUX_USE_CURRENT_RANGE=
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/2
QT_GRAPHICSSYSTEM_CHECKED=1
USER=hanye
LD_LIBRARY_PATH=/home/hanye/Downloads/TensorRT-6.0.1.5/lib:/home/hanye/opt/gcc7.5/lib64:/usr/local/cuda-10.1/targets/x86_64-linux/lib64:/usr/local/cuda/lib64:/home/hanye/Downloads/cudnn/cuda/lib64:/opt/BIOVIA/LicensePack/linux/lib:/home/hanye/intel/mkl/lib/intel64:/home/hanye/intel/compilers_and_libraries/linux/lib/intel64:/usr/local/lib64
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=01;36:.au=01;36:.flac=01;36:.mid=01;36:.midi=01;36:.mka=01;36:.mp3=01;36:.mpc=01;36:.ogg=01;36:.ra=01;36:.wav=01;36:.axa=01;36:.oga=01;36:.spx=01;36:*.xspf=01;36:
ACCELRYS_LICENSE_PLATFORM=linux
MSI_LICENSE_FILE=/opt/BIOVIA/LicensePack/Licenses:25100@localhost
VIRTUAL_ENV=/home/hanye/venv
MAIL=/var/spool/mail/hanye
PATH=/home/hanye/venv/bin:/home/hanye/deepmd-kit/bin/:/home/hanye/Downloads/TensorRT-6.0.1.5/bin:/home/hanye/opt/gcc7.5/bin/:/usr/local/cuda/bin/:/home/hanye/anaconda3/bin:/home/hanye/anaconda3/bin:/opt/BIOVIA/LicensePack/linux/bin:/usr/local/bin:/usr/local/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hanye/Downloads/p4vasp-0.3.30:/home/hanye/p4vasp/bin:/home/hanye/.local/bin:/home/hanye/bin
PWD=/home/hanye
LANG=en_US.UTF-8
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
LOADEDMODULES=
PS1=(venv) [\u@\h \W]$
SELINUX_LEVEL_REQUESTED=
ACCELRYS_LIC_PACK_DIR=/opt/BIOVIA/LicensePack
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/hanye
LOGNAME=hanye
QTLIB=/usr/lib64/qt-3.3/lib
XDG_DATA_DIRS=/home/hanye/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
SSH_CONNECTION=192.168.111.11 60735 192.168.41.101 22
MODULESHOME=/usr/share/Modules
LESSOPEN=||/usr/bin/lesspipe.sh %s
XDG_RUNTIME_DIR=/run/user/1000
DISPLAY=localhost:12.0
BASH_FUNC_module()=() { eval /usr/bin/modulecmd bash $*
}

0 replies

hanyecn · 2020-10-20T13:25:16Z

hanyecn
Oct 20, 2020
Author

I tried to downgrade cuda to 9.2, the error of "Segmentation fault (core dumped)" is still there. My environment variables are shown below:
(base) [dp9.2@hanye lmp]$ env
XDG_SESSION_ID=172
HOSTNAME=hanye
SELINUX_ROLE_REQUESTED=
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=192.168.111.11 50381 22
CONDA_SHLVL=1
CONDA_PROMPT_MODIFIER=(base)
SELINUX_USE_CURRENT_RANGE=
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/0
QT_GRAPHICSSYSTEM_CHECKED=1
USER=dp9.2
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=01;36:.au=01;36:.flac=01;36:.mid=01;36:.midi=01;36:.mka=01;36:.mp3=01;36:.mpc=01;36:.ogg=01;36:.ra=01;36:.wav=01;36:.axa=01;36:.oga=01;36:.spx=01;36:*.xspf=01;36:
CONDA_EXE=/home/dp9.2/anaconda3/bin/conda
_CE_CONDA=
MAIL=/var/spool/mail/dp9.2
PATH=/home/dp9.2/anaconda3/bin:/home/dp9.2/anaconda3/condabin:/usr/local/bin:/usr/local/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/dp9.2/.local/bin:/home/dp9.2/bin
CONDA_PREFIX=/home/dp9.2/anaconda3
PWD=/home/dp9.2/water/lmp
LANG=en_US.UTF-8
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
LOADEDMODULES=
SELINUX_LEVEL_REQUESTED=
_CE_M=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/dp9.2
CONDA_PYTHON_EXE=/home/dp9.2/anaconda3/bin/python
LOGNAME=dp9.2
QTLIB=/usr/lib64/qt-3.3/lib
XDG_DATA_DIRS=/home/dp9.2/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
SSH_CONNECTION=192.168.111.11 50381 192.168.41.101 22
MODULESHOME=/usr/share/Modules
CONDA_DEFAULT_ENV=base
LESSOPEN=||/usr/bin/lesspipe.sh %s
XDG_RUNTIME_DIR=/run/user/1006
DISPLAY=localhost:10.0
BASH_FUNC_module()=() { eval /usr/bin/modulecmd bash $*
}
_=/usr/bin/env
OLDPWD=/home/dp9.2/water(base)

Any suggestions? Thanks a lot.
BTW, the cpu version can works well for both conda10.1 and 9.2.

0 replies

njzjz · 2020-10-20T19:40:31Z

njzjz
Oct 20, 2020
Maintainer

I suggest to clean your LD_LIBRARY_PATH

unset LD_LIBRARY_PATH

0 replies

hanyecn · 2020-10-23T13:51:34Z

hanyecn
Oct 23, 2020
Author

Hi, I've tried to get this done on a new server, but the same error occurred. Now, I'm trying to change my OS for CentOs 7.0 to Ubuntu.

0 replies

njzjz · 2020-10-23T16:15:20Z

njzjz
Oct 23, 2020
Maintainer

Did you try our docker images? https://github.com/orgs/deepmodeling/packages/container/deepmd-kit/77709

0 replies

njzjz · 2020-10-24T00:57:14Z

njzjz
Oct 24, 2020
Maintainer

Which potential did you train, water.json or water_se_a.json? You can try another one.

0 replies

hanyecn · 2020-11-04T17:10:30Z

hanyecn
Nov 4, 2020
Author

I'm trying to train with water.json. docker images can't work on my machine, it gives me a lot of strange error messages, I will not try to use docker image any more. Now I'm using cpu version, but it's really slow.

0 replies

I've got the same error while compiling deepmd-kit #272

hanyecn Oct 8, 2020

Replies: 47 comments

njzjz Oct 9, 2020 Maintainer

hanyecn Oct 9, 2020 Author

njzjz Oct 9, 2020 Maintainer

hanyecn Oct 9, 2020 Author

hanyecn Oct 9, 2020 Author

njzjz Oct 9, 2020 Maintainer

hanyecn Oct 9, 2020 Author

hanyecn Oct 9, 2020 Author

hanyecn Oct 9, 2020 Author

njzjz Oct 9, 2020 Maintainer

hanyecn Oct 9, 2020 Author

hanyecn Oct 11, 2020 Author

includes

hanyecn Oct 11, 2020 Author

hanyecn Oct 11, 2020 Author

njzjz Oct 11, 2020 Maintainer

Mskater Oct 13, 2020

hanyecn Oct 14, 2020 Author

njzjz Oct 14, 2020 Maintainer

hanyecn Oct 14, 2020 Author

Mskater Oct 15, 2020

hanyecn Oct 15, 2020 Author

hanyecn Oct 18, 2020 Author

njzjz Oct 18, 2020 Maintainer

hanyecn Oct 19, 2020 Author

hanyecn Oct 20, 2020 Author

njzjz Oct 20, 2020 Maintainer

hanyecn Oct 23, 2020 Author

njzjz Oct 23, 2020 Maintainer

njzjz Oct 24, 2020 Maintainer

hanyecn Nov 4, 2020 Author

hanyecn
Oct 8, 2020

njzjz
Oct 9, 2020
Maintainer

hanyecn
Oct 9, 2020
Author

njzjz
Oct 9, 2020
Maintainer

hanyecn
Oct 9, 2020
Author

hanyecn
Oct 9, 2020
Author

njzjz
Oct 9, 2020
Maintainer

hanyecn
Oct 9, 2020
Author

hanyecn
Oct 9, 2020
Author

hanyecn
Oct 9, 2020
Author

njzjz
Oct 9, 2020
Maintainer

hanyecn
Oct 9, 2020
Author

hanyecn
Oct 11, 2020
Author

hanyecn
Oct 11, 2020
Author

hanyecn
Oct 11, 2020
Author

njzjz
Oct 11, 2020
Maintainer

Mskater
Oct 13, 2020

hanyecn
Oct 14, 2020
Author

njzjz
Oct 14, 2020
Maintainer

hanyecn
Oct 14, 2020
Author

Mskater
Oct 15, 2020

hanyecn
Oct 15, 2020
Author

hanyecn
Oct 18, 2020
Author

njzjz
Oct 18, 2020
Maintainer

hanyecn
Oct 19, 2020
Author

hanyecn
Oct 20, 2020
Author

njzjz
Oct 20, 2020
Maintainer

hanyecn
Oct 23, 2020
Author

njzjz
Oct 23, 2020
Maintainer

njzjz
Oct 24, 2020
Maintainer

hanyecn
Nov 4, 2020
Author