Generating pinvoke wrappers using ClangSharpPInvokeGenerator generates CXError_Failure errors #283

bdurrani · 2021-10-30T17:32:12Z

I'm trying to generate pinvoke wrappers for poppler cpp header files on ubuntu.

You can see the code for those those headers here
https://gitlab.freedesktop.org/poppler/poppler/-/blob/master/cpp/poppler-font.h

Poppler is installed on my system via apt

I'm running the generate tool as follows

~/.dotnet/tools/ClangSharpPInvokeGenerator @generate.rsp

The response file looks like the following

--config
compatible-codegen
generate-tests-xunit
multi-file
unix-types
log-potential-typedef-remappings
log-visited-files 
--file
poppler-font.h
poppler-document.h
poppler-page.h
poppler-toc.h
poppler-version.h
--namespace
Popper.Interop
--output
./output
--test-output
./tests
--include-directory
/usr/include/poppler/cpp
/usr/include/poppler
--language
c++

I'm using the 11.0.0-beta3 tool installed globally

 dotnet tool install --global ClangSharpPInvokeGenerator --version 11.0.0-beta3

Version of clang is the following

❯ clang --version                                                               
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Whenever I run the generator tool, I get the following errors

Error: Parsing failed for 'poppler-font.h' due to 'CXError_Failure'.
Skipping 'poppler-font.h' due to one or more errors listed above.

Error: Parsing failed for 'poppler-document.h' due to 'CXError_Failure'.
Skipping 'poppler-document.h' due to one or more errors listed above.

Error: Parsing failed for 'poppler-page.h' due to 'CXError_Failure'.
Skipping 'poppler-page.h' due to one or more errors listed above.

Error: Parsing failed for 'poppler-toc.h' due to 'CXError_Failure'.
Skipping 'poppler-toc.h' due to one or more errors listed above.

Error: Parsing failed for 'poppler-version.h' due to 'CXError_Failure'.
Skipping 'poppler-version.h' due to one or more errors listed above.

I'm trying to figure out why this tool is failing, but the errors don't tell me much.
Is there any way for me to get more information? Just trying to figure out what the next steps should be.

Thanks for your help.

The text was updated successfully, but these errors were encountered:

tannergooding · 2021-10-31T06:30:35Z

My expectation is the failure is caused by a version mismatch. You are running ClangSharpPInvokeGenerator v11.0.0-beta3, but the installed libClang is 10.0.0-4ubuntu1.

There is a strict requirement that the major version of libClang and the major version of ClangSharp match up (e.g. 11 to 11, 12 to 12, 13 to 13, etc). If you can install libClang v11 then I expect the errors will mostly resolve.

Aside from that, rolling forward to the latest nightly bits (and libClang v13) will bring along some general improvements for startup diagnostics, including telling you if the loaded version of libClang is the wrong version.

There isn't a NuGet available for the PInvokeGenerator yet.
Some other fixes are planned for v13 and an official release is delayed as we're in the process of trying to move to the .NET Foundation (ClangSharp and LLVMSharp dotnet-foundation/projects#184)

bdurrani · 2021-10-31T13:24:47Z

Thanks so much. I'll try this out with v12 and see how it goes

bdurrani · 2021-10-31T14:11:18Z

I installed clang v12 on ubuntu 20.04
I have libclang present. I installed in from the script at the following location
https://apt.llvm.org/

root@ea89bf12b378:/tmp# clang-12 --version
Ubuntu clang version 12.0.1-++20211029101322+fed41342a82f-1~exp1~20211029221816.4
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

I have libclang 12 installed as well.

root@ea89bf12b378:/tmp# ll /usr/lib/x86_64-linux-gnu/*clang*
lrwxrwxrwx 1 root root       16 Oct 29 18:18 /usr/lib/x86_64-linux-gnu/libclang-12.so -> libclang-12.so.1
-rw-r--r-- 1 root root 26290256 Oct 29 18:18 /usr/lib/x86_64-linux-gnu/libclang-12.so.1
lrwxrwxrwx 1 root root       33 Oct 29 18:18 /usr/lib/x86_64-linux-gnu/libclang-cpp.so.12 -> ../llvm-12/lib/libclang-cpp.so.12

I installed the v12 version of the pinvoke generate tool.

dotnet tool install --global ClangSharpPInvokeGenerator --version 12.0.0-beta2

Unfortunately, same issue

root@ea89bf12b378:/tmp# ClangSharpPInvokeGenerator @generate.rsp
Error: Parsing failed for 'poppler_cpp_export.h' due to 'CXError_Failure'.
Skipping 'poppler_cpp_export.h' due to one or more errors listed above.

I was getting errors about missing libclang but those went away once I installed explicitly via the following

apt install --yes libclang-12-dev
root@ea89bf12b378:/tmp# apt install --yes libclang-12-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  libclang-12-dev
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 19.8 MB of archives.
After this operation, 194 MB of additional disk space will be used.
Get:1 https://apt.llvm.org/focal llvm-toolchain-focal-12/main amd64 libclang-12-dev amd64 1:12.0.1~++20211029101322+fed41342a82f-1~exp1~20211029221816.4 [19.8 MB]
Fetched 19.8 MB in 4s (5291 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libclang-12-dev.
(Reading database ... 69626 files and directories currently installed.)
Preparing to unpack .../libclang-12-dev_1%3a12.0.1~++20211029101322+fed41342a82f-1~exp1~20211029221816.4_amd64.deb ...
Unpacking libclang-12-dev (1:12.0.1~++20211029101322+fed41342a82f-1~exp1~20211029221816.4) ...
Setting up libclang-12-dev (1:12.0.1~++20211029101322+fed41342a82f-1~exp1~20211029221816.4) ...

Thanks again for your help

bdurrani · 2021-10-31T18:21:12Z

As a test, I decided to try a really simple header, so see if I have things set up properly

root@ea89bf12b378:/tmp# cat test.h
/* Defines the poppler version. */
#ifndef POPPLER_VERSION
#define POPPLER_VERSION "0.86.1"
#endif

Looks like libClangSharp is missing. Is this a separate step that I missed?
I had assumed installing the pinvoke tool would have installed it for me.

root@ea89bf12b378:/tmp# ClangSharpPInvokeGenerator @generate.rsp
Processing 'test.h'
Diagnostics for binding generation:
    Error: System.DllNotFoundException: Unable to load shared library 'libClangSharp' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibClangSharp: cannot open shared object file: No such file or directory
   at ClangSharp.Interop.clangsharp.Cursor_getDeclKind(CXCursor C)
   at ClangSharp.Interop.CXCursor.get_DeclKind() in /_/sources/ClangSharp.Interop/Extensions/CXCursor.cs:line 522
   at ClangSharp.Decl.Create(CXCursor handle) in /_/sources/ClangSharp/Cursors/Decls/Decl.cs:line 148
   at ClangSharp.Cursor.Create(CXCursor handle) in /_/sources/ClangSharp/Cursors/Cursor.cs:line 73
   at ClangSharp.TranslationUnit.GetOrCreate[TCursor](CXCursor handle) in /_/sources/ClangSharp/TranslationUnit.cs:line 102
   at ClangSharp.TranslationUnit.<.ctor>b__9_0() in /_/sources/ClangSharp/TranslationUnit.cs:line 34
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at ClangSharp.PInvokeGenerator.GenerateBindings(TranslationUnit translationUnit, String filePath, String[] clangCommandLineArgs, CXTranslationUnit_Flags translationFlags) in /_/sources/ClangSharp.PInvokeGenerator/PInvokeGenerator.cs:line 316

Sorry for all the questions.

tannergooding · 2021-10-31T20:42:47Z

Sorry for all the questions.

No worries, not all of this is super clear today. Hopefully the issues can get resolved and the tooling can move to a place where its much simpler to use.

Looks like libClangSharp is missing. Is this a separate step that I missed?
I had assumed installing the pinvoke tool would have installed it for me.

It's supposed to, but there are some issues with native dependencies and NuGet that make this a difficult experience and it doesn't always work (#229 (comment) goes into some more detail).

You can manually get the binary from https://www.nuget.org/packages/libClangSharp.runtime.ubuntu.20.04-x64/12.0.0-beta1 and put it next to the tool (ClangSharpPInvokeGenerator) which should allow it to be resolved.

bdurrani · 2021-10-31T21:26:20Z

Sigh, no luck.

I did copy libClangSharp.so but I think one of it's dependencies is missing

root@ea89bf12b378:~/.dotnet/tools# ldd libClangSharp.so
        linux-vdso.so.1 (0x00007ffc1ac65000)
        libclang.so.12 => /lib/x86_64-linux-gnu/libclang.so.12 (0x00007f664bf45000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f664bf22000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f664bf06000)
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f664bed6000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f664bcf4000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f664bba5000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f664bb88000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f664b996000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f664feee000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f664b990000)
        libLLVM-12.so.1 => /lib/x86_64-linux-gnu/libLLVM-12.so.1 (0x00007f6645e75000)
        libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f6645e69000)
        libedit.so.2 => /lib/x86_64-linux-gnu/libedit.so.2 (0x00007f6645e2f000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6645e24000)
        libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f6645c6a000)
        libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f6645c50000)
        libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007f6645a6a000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f6645a41000)
        libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007f6643f7e000)

I tried to soft symlink libclang to use the name it expects, but there must be something else I'm missing

root@ea89bf12b378:~/.dotnet/tools# ll /lib/x86_64-linux-gnu/libclang*
lrwxrwxrwx 1 root root       16 Oct 29 18:18 /lib/x86_64-linux-gnu/libclang-12.so -> libclang-12.so.1
-rw-r--r-- 1 root root 26290256 Oct 29 18:18 /lib/x86_64-linux-gnu/libclang-12.so.1
lrwxrwxrwx 1 root root       33 Oct 29 18:18 /lib/x86_64-linux-gnu/libclang-cpp.so.12 -> ../llvm-12/lib/libclang-cpp.so.12
lrwxrwxrwx 1 root root       16 Oct 31 17:14 /lib/x86_64-linux-gnu/libclang.so.12 -> libclang-12.so.1
lrwxrwxrwx 1 root root       16 Oct 31 17:19 /lib/x86_64-linux-gnu/libclang.so.12.so -> libclang-12.so.1

ah well...

tannergooding · 2021-10-31T22:22:55Z

Hmmm. I just tried on a clean machine and everything looks to work as expected.

ClangSharp 13.0.0

I did the following

sudo apt install git gcc clang-13 llvm-13-dev
git clone https://github.com/microsoft/clangsharp
cd ./clangsharp
OVERRIDE_RUNTIME_IDENTIFIER=ubuntu.20.04-x64
./scripts/cibuild.sh --ci

That failed, so I then did:

sudo cp ~/.nuget/packages/libclangsharp.runtime.ubuntu.20.04-x64/13.0.0-beta1/runtimes/ubuntu.20.04-x64/native/libClangSharp.so /usr/lib/x86_64-linux-gnu/libClangSharp.so
./scripts/cibuild.sh --ci

And everything succeeded.

ClangSharp 12.0.0

I did the following:

sudo apt install git gcc clang-12 llvm-12-dev
install .NET 5: https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#2004-
dotnet tool install --global ClangSharpPInvokeGenerator --version 12.0.0-beta2
download and extract https://www.nuget.org/packages/libClangSharp.runtime.ubuntu.20.04-x64/12.0.0-beta1
sudo cp ~/downloads/libclangsharp.runtime.ubuntu.20.04-x64/12.0.0-beta1/runtimes/ubuntu.20.04-x64/native/libClangSharp.so /usr/lib/x86_64-linux-gnu/libClangSharp.so
echo "void M();" > Test.h
ClangSharpPInvokeGenerator --file Test.h -n Test -o ./Test.cs

That failed with the type load error, so I did:

ldd /usr/lib/x86_64-linux-gnu/libClangSharp.so
- This shows libclang.so.12 couldn't be resolved, as you saw
sudo ln -s /usr/lib/x86_64-linux-gnu/libclang-12.so.1 /usr/lib/x86_64-linux-gnu/libclang.so.12
ClangSharpPInvokeGenerator --file Test.h -n Test -o ./Test.cs

And everything succeeded.

bdurrani · 2021-10-31T22:45:25Z

doh 🤦🏽
I had copied libClangSharp.so to ~/.dotnet/tools.
After installing it to /usr/lib/x86_64-linux-gnu/, it works with the simple case.

I guess I'll have to try and simplify why the popper headers fail to parse. Now that I know the basic set up works.

Thanks again for your help with this

Would you be ok if I submitted a PR for the README with some of these steps in there? Or do you think this would change too often to be useful?

bdurrani · 2021-10-31T22:52:17Z

Another brain fart...
It looks like the wrapper generation works for the headers if I give it the fully qualified path 🤦🏽‍♂️
I guess CXError_Failure in this case meant "Could not find the file"
sigh....

tannergooding · 2021-10-31T23:03:38Z

Would you be ok if I submitted a PR for the README with some of these steps in there? Or do you think this would change too often to be useful?

I'm definitely fine with some additional hints in the docs or even in the failure case on the command line.

Most of the time right now we only expose what libClang itself reports, which is often less than helpful.

tannergooding closed this as completed Sep 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generating pinvoke wrappers using ClangSharpPInvokeGenerator generates CXError_Failure errors #283

Generating pinvoke wrappers using ClangSharpPInvokeGenerator generates CXError_Failure errors #283

bdurrani commented Oct 30, 2021

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021 •

edited

Loading

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

tannergooding commented Oct 31, 2021

Generating pinvoke wrappers using ClangSharpPInvokeGenerator generates CXError_Failure errors #283

Generating pinvoke wrappers using ClangSharpPInvokeGenerator generates CXError_Failure errors #283

Comments

bdurrani commented Oct 30, 2021

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021 • edited Loading

tannergooding commented Oct 31, 2021

ClangSharp 13.0.0

ClangSharp 12.0.0

bdurrani commented Oct 31, 2021

bdurrani commented Oct 31, 2021

tannergooding commented Oct 31, 2021

bdurrani commented Oct 31, 2021 •

edited

Loading