ImportError: libGymFactory.so: cannot open shared object file: No such file or directory

[PierreExeter]

Hello,

I installed gym-ignition in developer mode, following the instructions on the README. There wasn't any error during the compilation. However when I try to run the example given with :

python3 /path/to/gym-ignition-repo/examples/python/launch_cartpole.py

the libGymFactory.so seems to be missing.

This is the error I get:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gympp_bindings.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/home/pierre/bin/anaconda3/envs/gym-ignition/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: libGymFactory.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "launch_cartpole.py", line 14, in <module>
    import gym_ignition
  File "gym-ignition/gym_ignition/__init__.py", line 11, in <module>
    import gympp_bindings
  File "/usr/local/lib/python3.6/site-packages/gympp_bindings.py", line 17, in <module>
    _gympp_bindings = swig_import_helper()
  File "/usr/local/lib/python3.6/site-packages/gympp_bindings.py", line 16, in swig_import_helper
    return importlib.import_module('_gympp_bindings')
  File "/home/pierre/bin/anaconda3/envs/gym-ignition/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libGymFactory.so: cannot open shared object file: No such file or directory

How can I solve this?

My system:
Ubuntu 18.04 bionic
Python 3.6
cmake version 3.16.2

Note: I used Anaconda instead of virtualenv to create the virtual environment as indicated in the instructions but I don't think this is what's causing the issue.

Thanks

[traversaro]

Hi @PierreExeter,
I am not an expert of those instructions, but I think I can provide you some useful info nevertheless.
TL;DR :

• The "User setup" installation using pip3 install gym-ignition is working fine? There is any reason why you want to install the library with the "developer setup"?
• In which "install prefix" you installed the CMake project in the cmake -DCMAKE_INSTALL_PREFIX=<installprefix> .. step?
• Can you try to just use virtualenv instead of Anaconda, as documented in installation instructions?

Long explanation:
gym-ignition is not a pure Python-package, but is rather a package that heavily relies on libraries available on your system to work, in particular the libraries provided by the Ignition Robotics project ( https://ignitionrobotics.org/), in particular Ignition Gazebo. In the installation instructions, this libraries are installed in a system prefix (/usr) using apt-get, the Ubuntu system manager.

On the other hand, Anaconda is a complete environment that ships its own version of several packages that apt-get install as well, including its own version of the Python interpreter. Mixing system packages and anaconda packages is not a good idea in general, as you do not know if they are compatible with one another (as you could have the same library in both the system and anaconda, but at two different versions). For more on this, see https://www.anaconda.com/using-pip-in-a-conda-environment/ .

If anyone wants to use gym-ignition in anaconda, my personal suggestion is to install all the C++ dependencies from conda-forge, and build the one that are not available from conda-forge from source.

To streamline this process and have a conda install gym-ignition it would be necessary is to create a real gym-ignition conda package (so-called "feedstock") and contribute it to a repo of conda packages such as conda-forge. The main problem is that you would also need to package all the dependencies of this project in conda-forge as well, including all the ignition libraries at the right version, see conda-forge/staged-recipes#10604 for an example of a contributed conda-forge package. As you may imagine, this may involve a non-trivial amount of work. fyi @seanyen

[PierreExeter]

Hi @traversaro,

Thanks for your answer and for helping maintaining this code.

• Yes the "User setup" is working fine. I can run the example given. The reason why I want to install the "Developer setup" is that I want to create my own Gym environment, not only the examples provided. I have already created this custom Gym environment in the PyBullet simulator (from a .urdf geometry file) but I want to use Gazebo instead, hence my need for gym-ignition. Is it possible to create custom Gym environment with the "User setup"?
• I installed the Cmake project in /usr/local as recommended.
• I tried with virtualenv but I must have broken something on my machine because I cannot build using Cmake anymore (either with a virtualenv or an Anaconda environment), see error file attached.
  cmake_error.txt

Anyway, I will look for another solution to create my custom Gym environment with the Gazebo engine. Thanks again for your help.

[traversaro]

* I tried with virtualenv but I must have broken something on my machine because I cannot build using Cmake anymore (either with a virtualenv or an Anaconda environment), see error file attached.
  [cmake_error.txt](https://github.com/robotology/gym-ignition/files/4071972/cmake_error.txt)

Why are you running cmake --build . as sudo? In any case, the error seems due to the fact that gcc-8 is not used for compilation, are you sure that CC and CXX enviroment variables have the correct values documented in step 2 of https://github.com/robotology/gym-ignition#developer-setup ?

[PierreExeter]

I need to run cmake -DCMAKE_INSTALL_PREFIX=/usr/local .. as sudo because it complains about permissions being denied otherwise (I guess because /usr/local has restricted permissions). I tried cmake --build . without sudo but I get the same error.

I also tried to set the Cmake project in a directory with full permission

cmake -DCMAKE_INSTALL_PREFIX=~/gym-ignition/cmake_project ..

But I get the following error:
cmake_prefix_error.txt
CMakeError.log
CMakeOutput.log

Yes I believe that CC and CXX are properly set, see the output below.

$ echo $CC
gcc-8
$ echo $CXX
g++-8

[traversaro]

I need to run cmake -DCMAKE_INSTALL_PREFIX=/usr/local .. as sudo because it complains about permissions being denied otherwise (I guess because /usr/local has restricted permissions).

The CMake configuration does not copy anything in /usr/local, the only step that copies files in /usr/local is the install, and that is the only one that needs to be run as sudo. The cmake and cmake --build . fail to run without sudo probably because you run them as sudo at some point, and so some of the build files need sudo to be accessed. If you try to remove the build directory and start from scratch, you should not need to run cmake under sudo, and sudo will be necessary only for the installation under /usr/local .

Inspecting your log, it seems that the CMake failure in that case is due to this:

CMake Error at /usr/share/dart/cmake/DARTFindurdfdom.cmake:13 (set_target_properties):
  Property INTERFACE_LINK_LIBRARIES may not contain link-type keyword
  "optimized".  The INTERFACE_LINK_LIBRARIES property may contain
  configuration-sensitive generator-expressions which may be used to specify
  per-configuration rules.

This could be a regression in some of the system dependencies, the CI apparently does not have periodic tests running (cc @diegoferigo), so it is possible that we missed that, I opened #119 to see if we can reproduce the error.

[traversaro]

I can't reproduce the error, and my urdfdom is installed from the system, see:

straversaro@iiticublap103:~/src/gym-ignition/build2$ apt show liburdfdom-dev 
Package: liburdfdom-dev
Version: 1.0.0-2ubuntu0.1
Priority: extra
Section: universe/libdevel
Source: urdfdom
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 30.7 kB
Depends: libtinyxml-dev, libconsole-bridge-dev, liburdfdom-headers-dev (>= 1.0), liburdfdom-model (= 1.0.0-2ubuntu0.1), liburdfdom-model-state (= 1.0.0-2ubuntu0.1), liburdfdom-sensor (= 1.0.0-2ubuntu0.1), liburdfdom-world (= 1.0.0-2ubuntu0.1)
Homepage: https://github.com/ros/urdfdom
Download-Size: 4,368 B
APT-Manual-Installed: no
APT-Sources: http://it.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages
Description: URDF DOM - development files
 The URDF (U-Robot Description Format) library provides core data
 structures and a simple XML parsers for populating the class data
 structures from an URDF file.
 .
 This package contains the development files (headers, pkg-config and
 CMake files).

N: There is 1 additional record. Please use the '-a' switch to see it

What is the output of apt show liburdfdom-dev in your case?

[PierreExeter]

This is my output of apt show liburdfdom-dev:

$ apt show liburdfdom-dev
Package: liburdfdom-dev
Version: 1.0.0-2ubuntu0.1
Priority: extra
Section: universe/libdevel
Source: urdfdom
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 30.7 kB
Depends: libtinyxml-dev, libconsole-bridge-dev, liburdfdom-headers-dev (>= 1.0), liburdfdom-model (= 1.0.0-2ubuntu0.1), liburdfdom-model-state (= 1.0.0-2ubuntu0.1), liburdfdom-sensor (= 1.0.0-2ubuntu0.1), liburdfdom-world (= 1.0.0-2ubuntu0.1)
Homepage: https://github.com/ros/urdfdom
Download-Size: 4,368 B
APT-Manual-Installed: yes
APT-Sources: http://ie.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages
Description: URDF DOM - development files
 The URDF (U-Robot Description Format) library provides core data
 structures and a simple XML parsers for populating the class data
 structures from an URDF file.
 .
 This package contains the development files (headers, pkg-config and
 CMake files).

N: There is 1 additional record. Please use the '-a' switch to see it

The only difference is APT-Manual-Installed: yes

[traversaro]

Can you share the CMakeCache.txt of your failed build? I suspect that there is some other DART or urdfdom installed in your system, but I am not sure about this.

[PierreExeter]

CMakeCache.txt

[PierreExeter]

I also did a fresh install (i.e. deleted the virtualenv, deleted the cloned github directory and followed the steps on the README from the beginning). The same error occurs when running cmake -DCMAKE_INSTALL_PREFIX=/usr/local ..

CMake Error at /usr/share/dart/cmake/DARTFindurdfdom.cmake:13 (set_target_properties)

fresh_install.txt

[PierreExeter]

I tried on another machine and I was able to build without error so the problem comes from my laptop.
(However now it's the Python module gympp_bindings that's missing...)
Anyway, thanks again for your help.

[diegoferigo]

Thanks @traversaro for taking care of this issue. I am currently AFK, will be back next week and I will have a deeper look to the problem.

In the meantime I want to point out that downstream users like @PierreExeter are not required to install the project in developer mode to create environments. From the point of view of this project, developer means someone that wants to improve gym-ignition. Users of this project, instead, are all those that want to create environments with it.

Probably we should make this even more clear in the documentation.

[traversaro]

I also did a fresh install (i.e. deleted the virtualenv, deleted the cloned github directory and followed the steps on the README from the beginning). The same error occurs when running cmake -DCMAKE_INSTALL_PREFIX=/usr/local ..

CMake Error at /usr/share/dart/cmake/DARTFindurdfdom.cmake:13 (set_target_properties)

fresh_install.txt

Beside what @diegoferigo correctly pointed out on the possibility of creating enviroments also with the pip-installed version of gym-ignition, I think the problem of these setup lies in the fact that there is a urdfdom installed from ROS2 that shadows the system one, and that the CMake config of that urdfdom version is not compatible with DART's CMake logic, as highlithed by the following lines of the CMakeCache.txt :

//The directory containing a CMake configuration file for urdfdom.
urdfdom_DIR:PATH=/opt/ros/dashing/lib/urdfdom/cmake

//The directory containing a CMake configuration file for urdfdom_headers.
urdfdom_headers_DIR:PATH=/opt/ros/dashing/lib/urdfdom_headers/cmake

I guess that you have some sort of source /opt/ros/dashing/setup.bash in your .bashrc or in a similar location. If you remove it and create a new build, the build should be successful, so you will be able to install gym-ignition in both user and developer mode.

[PierreExeter]

@traversaro you were right, I had this line in my .bashrc from another install:

source ~/ros2learn/environments/gym-gazebo2/provision/mara_setup.sh

Once removed, I was able to create a new build. Sorry for all the hassle.

In the meantime I want to point out that downstream users like @PierreExeter are not required to install the project in developer mode to create environments.

Ok I didn't realise this was possible in "User mode". Is there a documentation or some instructions available to create custom Gym environments? From what I understand, Gym environments are registered in:

~/venv/lib/python3.6/site-packages/gym_ignition/__init__.py

Is there any documentation on the register() function (in particular, on how to define the arguments task_cls and robot_cls)?

[diegoferigo]

@PierreExeter

We are in the process of releasing the first stable release, you can expect it in the next weeks (see #87). We will provide also tutorial on how to create environments in an external project that imports gym-ignition, adding your own environments inside this repository is discouraged.

For creating custom and standalone environments, you can refer to the official documentation. The only difference is you have to structure your files in a similar way as the provided cartpole environment (defining a task and a robot object). For more details refer to:

• Registration: __init__.py
• Model: CartPole.urdf
• Robot object: robots/sim/gazebo/cartpole.py
• Task: tasks/cartpole_discrete.py

Then, if you want to understand under the hood what's happening, you can also check the following classes (you don't need to edit them):

• robots.base.GazeboRobot
• runtimes.GazeboRuntime

Finally, if you still didn't read the project's paper, you can find it on ArXiv.

[diegoferigo]

Closing since the original problem has been solved.