Missing dependency for GPU

[gmseabra]

Hi,

First of all, I want to thank you guys for making this available for the community. This is a great step forward in molecular generation.

However, I noticed a slight problem when using GPU. After strictly following the steps in the installation, and saving the pretrained model in the PRETRAINED_MODEL dir, I got the results below. It does still work, but there's something weird going on with the Tensorflow.

I tried installing tensorflow-gpu==2.1.0, but the message persists. What am I missing?

I'm working on a RHEL8 station with CUDA-10.2.

All the best,
Gustavo.

$ molecule_generation sample PRETRAINED_MODEL 10
2022-05-04 16:12:42.673853: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/amber/current/lib:/usr/local/cuda-10.2/lib64:/opt/amber/current/lib:/usr/local/cuda-10.2/lib64::/usr/lib64/openmpi/lib:/usr/lib64/openmpi/lib
2022-05-04 16:12:42.673995: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/amber/current/lib:/usr/local/cuda-10.2/lib64:/opt/amber/current/lib:/usr/local/cuda-10.2/lib64::/usr/lib64/openmpi/lib:/usr/lib64/openmpi/lib
2022-05-04 16:12:42.674011: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Loading a trained model from: PRETRAINED_MODEL/GNN_Edge_MLP_MoLeR__2022-02-24_07-16-23_best.pkl
2022-05-04 16:12:44,365 trace_dataset.py:44 INFO Initialising TraceDataset.
2022-05-04 16:12:44,392 trace_dataset.py:44 INFO Initialising TraceDataset.
2022-05-04 16:12:44,403 trace_dataset.py:44 INFO Initialising TraceDataset.
2022-05-04 16:12:44,412 trace_dataset.py:44 INFO Initialising TraceDataset.
2022-05-04 16:12:44,423 trace_dataset.py:44 INFO Initialising TraceDataset.
2022-05-04 16:12:44,434 trace_dataset.py:44 INFO Initialising TraceDataset.
O=C1C2=CC=C(C3=CC=CC=C3)C=C=C2OC2=CC=CC=C12
CC(=O)NC1=NC2=CC(OCC3=CC=CN(CC4=CC=C(Cl)C=C4)C3=O)=CC=C2N1
CCN1C(=O)C2=CC=CC=C2N=C1NC(C)C(=O)NCC(=O)N=[N+]=[N-]
CC(=O)N1CCCC1C1=NC2=CC=C(C(C)(C)CCCC(C)C)C=C2NC1=NC1=CC=C(O)C=C1
N=C(N)NCCCCOC1=CC=C(Br)C(Cl)=N1
O=C1C2=CC=C(C3=NN=CO3)C=C2N=CN1CC1=CC=C(C2=CC=CC=C2)C=C1
O=CCCCCCN1C=CC2=CC=CC=C21
CCOC(=O)C1=CC2=CC(CC(C)C)=CC=C2N=C1C1=CC=C(Br)C=C1
CC1=C(C#N)C=C(NC(=O)NC2CCCCC2)N1CC#N
CC1=CNN=C1NC(=O)COC1=CC=C(Cl)C=C1Cl

[kmaziarz]

Hi Gustavo, thanks for reaching out!

What you're seeing are warnings, not errors (see this explanation), and they don't prevent you from running on GPU. I'm also getting the same warnings on my machine, and things work fine, giving the same output SMILES as what you pasted above. I think if you installed the TensorRT libraries separately this would go away, but I don't think we're actively trying to make use of TensorRT, so I would be surprised if installing it made something faster.

You can verify GPU is being used by drawing more samples (e.g. 1000), and running watch -n0.1 nvidia-smi in another terminal; if it's showing some GPU utilization then everything is likely fine.

The unexpected bit here is that we intended to turn these warnings off (see supress_tensorflow_warnings in utils/cli_utils.py), but it seems that didn't work. I made a separate issue to track this (#19).

[gmseabra]

I see now. Thanks!