Investigate bindgen woes - "file not found"

[Emilgardis]

Some (if not most?) *-dev packages on ubuntu dist only include headers in /usr/include, this means that the BINDGEN_EXTRA_CLANG_ARGS_<target>=--sysroot=$CROSS_SYSROOT that we set in our images means bindgen can't properly find these headers if the *-sys-crate doesn't use pkg-config probing.

bindgen errors with: Unable to generate bindings: ClangDiagnostic("wrapper.h:1:10: fatal error: 'mylib.h' file not found\n")

To solve this problem, there's two options, nr.1 requires a change in the sys-crate

1. Modify the sys-crate

Crates using bindgen for API generation can do the following

std::env::set_var("PKG_CONFIG_ALLOW_SYSTEM_CFLAGS", "1");
let library = pkg_config::probe_library("mylib").expect("Unable to probe `mylib`");
let bindings = bindgen::Builder::default()
    .clang_args(library.include_paths.iter().map(|path| format!("-I{}", path.to_string_lossy())))
    .header("wrapper.h")
    .parse_callbacks(Box::new(bindgen::CargoCallbacks))
    .generate()
    .expect("Unable to generate bindings");

2. modify BINDGEN_EXTRA_CLANG_ARGS_<target> to include /usr/include

(make sure to get the correct sysroot path and use the correct target names, <target> is lowercase snake_case target triple, <TARGET> is uppercase kebab-case target triple.)

# Cross.toml
[target.<TARGET>.env]
passthrough = ["BINDGEN_EXTRA_CLANG_ARGS_<target>=--sysroot=<path_to_target_sysroot> -idirafter/usr/include"]

Going forward

I've filed rust-lang/rust-bindgen#2701 to potentially make option 2 easier, as we then could just set BINDGEN_EXTRA_CLANG_ARGS=-idirafter/usr/include for all targets and all is well.

Can we solve this problem in our images?

Related: rust-lang/rust-bindgen#1247 for pkg-config with bindgen.

[Emilgardis]

Can we solve this problem in our images?

Should we add -I/usr/include in all our BINDGEN_EXTRA_CLANG_ARGS_<target>s maybe? One risk I see with this is that we introduce contamination, where the header files could potentially be different for the host architecture and the target architecture. How does clang prioritize the include dirs, right to left?

[Emilgardis]

BINDGEN_EXTRA_CLANG_ARGS_<target>=--sysroot=$CROSS_SYSROOT -I/usr/include -v

gives

#include <...> search starts here:
 /usr/include
 /usr/include/clang/10.0.0/include
 $CROSS_SYSROOT/include

clang says

For C++ inputs, if there are multiple -I options, these directories are searched in the order they are given before the standard system directories are searched.

BINDGEN_EXTRA_CLANG_ARGS_<target>=--sysroot=$CROSS_SYSROOT -nostdinc -I$CROSS_SYSROOT/include -I/usr/include works, but disables the std, there's no way to reenable it... we need -ibuiltininc (available on clang-11 and up) I think but that also doesn't to enable the clang includes

#include <...> search starts here:
 $CROSS_SYSROOT/include
 /usr/include

**edit

-idirafter/usr/include is exactly what we want

aarch64-unknown-linux-gnu example

# Cross.toml
[target.aarch64-unknown-linux-gnu.env]
passthrough = ["""BINDGEN_EXTRA_CLANG_ARGS_aarch64_unknown_linux_gnu=--sysroot=/usr/aarch64-linux-gnu -idirafter/usr/include"""]

gives

#include <...> search starts here:
 /usr/lib/llvm-11/lib/clang/10.0.0/include
 /usr/aarch64-linux-gnu/include
 /usr/include

I've updated the first post from using -I/usr/include to be -idirafter/usr/include

[Emilgardis]

Should we add -I/usr/include in all our BINDGEN_EXTRA_CLANG_ARGS_<target>s maybe? One risk I see with this is that we introduce contamination, where the header files could potentially be different for the host architecture and the target architecture. How does clang prioritize the include dirs, right to left?

I think -idirafter/usr/include is the way to do this, I don't see much hindering us from adding this to our images

[Emilgardis]

Some more investigating, pkg-config strips -I/usr/include as it's a system include path, setting PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 resolves this, I've modified the code to reflect this. I think this is wanted behaviour for the pkg_config crate

root@8f4378f92f97:/# PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config libssl --cflags --libs
-I/usr/include -L/usr/lib/aarch64-linux-gnu -lssl
root@8f4378f92f97:/# pkg-config libssl --cflags --libs
-L/usr/lib/aarch64-linux-gnu -lssl

[Emilgardis]

Another issue, should we really use PKG_CONFIG_PATH? From what I can gather, the expected thing to do is to use aarch64-linux-gnu-pkg-config, which discovers the path automatically. It also curiously omits -I/usr/include, very interesting...

[ydirson]

Some (if not most?) *-dev packages on ubuntu dist only include headers in /usr/include, this means that the BINDGEN_EXTRA_CLANG_ARGS_<target>=--sysroot=$CROSS_SYSROOT

I'm not entirely sure of the problem this issue is about, since not all containers use the same approach. Since you mention using Ubuntu's -dev packages, I assume you're talking about containers for targets using dpkg's foreign-architecture support, but then it seems to be the sysroot is just / so there should be no problem to start with. So I'm basically confused :)

[Emilgardis]

Maybe I'm the one that is confused, we set the env var in #707

I was under the impression that it was needed, but maybe it wasn't needed for every target, and only needed for android

[tniessen]

bindgen can't properly find these headers if the *-sys-crate doesn't use pkg-config probing

@Emilgardis In my case, I'm using bindgen with some C header files that do not belong to any pre-installed library, so as far as I know, I cannot reasonably use pkg-config. However, with default configuration only, this results in errors such as fatal error: 'stddef.h' file not found. (The reason is that the Rust crate needs to be aware of the memory layout of some C data structures, but it does not actually link against any C library.)

This works just fine with the default toolchain/target, but cross-compiling without cross requires manual modifications of the include path. Is there a more elegant solution now that I'm trying to use cross?

[Emilgardis]

@tniessen short answer is no, cross wouldnt help with a more elegant solution, the solution would be equally applicable with or without cross. I dont think your specific example here is related to what this issue is about, but feel free to open a new issue with more information and maybe we can work something out

[Emilgardis]

I'll close this issue with #1391