Undefined symbol on Python import

[vsoch]

Today is my first day using shroud - apologies if I'm hugely missing something!

I've created a yaml configuration file and I've gotten everything to work to run shroud, and install the Python library. But when I get to the moment of truth to import...

In [1]: import abispack
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-3fb25805f954> in <module>
----> 1 import abispack

ImportError: /usr/local/lib/python3.8/dist-packages/abispack-0.0.0-py3.8-linux-x86_64.egg/abispack.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN8abispack10Libabigail10GetVersionEv

oh no! Could you point me in the right direction for why the symbol is not defined? You can see all my code here

I noticed issue #39, and if I can figure this out for my own little Python project I could offer to help with a dummy example for here. I also saw that json isn't supported: https://github.com/LLNL/shroud/blob/develop/shroud/main.py#L439. Is this something that would be of interest? That would be fairly easy to add a function to load there.

[vsoch]

It does look like it's defined in the .so file:

<function-decl name='GetVersion' mangled-name='_ZN8abispack10Libabigail10GetVersionEv' filepath='abispack/../include/abispack.h' line='14' column='1' visibility='default' binding='global' size-in-bits='64'>

I guess what isn't clear to me is how shroud knows about the .cpp file if I've just provided the header file? It might be a different issue, but I'm curious.

[vsoch]

Stayed up all night - figured this out. The .hpp and .cpp files must be in the same top level directory, flattened - likely there is some auto-discovery going on when they are in the same path? You can look at my updated repository to see what works.

I'm hoping there can be support for files in different locations - small libraries it's okay to have them together, but I'm used to seeing header files in include and source in src. There also should be a note about this somewhere to save some future person like myself a missing night of sleep, haha.

[ltaylor16]

Shroud generates the wrappers from the YAML file. It does not read any source or header files. The YAML file is created by the user by copying declarations into the YAML for functions to be wrapped. For Python wrappers it also creates a setup.py file. This only includes the files that Shroud generates, since there can be several, and the user is expected to use it as a template to create their own. It does not include the source or dependent libraries for the library that is being wrapped.

In your case, I think you could keep your source as src/abispack.cpp by using that path in the setup.py.

I notice that you have to move your own version of setup.py out of the way before running Shroud to avoid overwriting it. Would it be useful to have an option to skip creating the setup.py file or maybe giving it a different name? The way we typically use Shroud is to have the YAML file in one directory then create the files in a different directory with the --outdir option. This would create the setup.py file in a different location as well.

I can add some comments to the generated setup.py file to indicate that it is intended as a starting point and does not know about any of the user's source files.

I think I saw a comment earlier about file name suffixes. You can set options to name the generated files whatever you want. The simplest fix is to change suffixes with some format fields at the top level:

format:
  C_header_filename_suffix: hh
  C_impl_filename_suffix: cc

[vsoch]

That sounds correct - I eventually figured out I needed to add it to sources but (if I remember correctly) there was still issue having the files in different folders. It would be hugely helpful I think to, given that a setup.py already exists, just not write it again. If the user wants to re-write it they can delete the old one.

I did use the Python-outdir argument and the setup.py was produced in the root - I actually think it’s ok because that’s how Python packages are typically structured (having that file outside the main module).

Let me know if you want any help with PRs or just want to quickly do it yourself! I’m definitely enjoying using shroud, although I haven’t gotten to more complicated use cases yet (I wanted to set up a simple working version before proceeding).