Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x86_64-pc-windows-gnu build failed #428

Closed
zonyitoo opened this issue May 25, 2024 · 11 comments · Fixed by #429
Closed

x86_64-pc-windows-gnu build failed #428

zonyitoo opened this issue May 25, 2024 · 11 comments · Fixed by #429

Comments

@zonyitoo
Copy link

Problem:

Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/win-x86_64/crypto/ to /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto
  /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto/fipsmodule/aesni-gcm-avx512.asm:10: fatal: unable to open include file `openssl/boringssl_prefix_symbols_nasm.inc'

It could be reproduced in this project: https://github.com/shadowsocks/shadowsocks-rust by running:

cross build --release --target x86_64-pc-windows-gnu --features "full"

Relevant details

AWS-LC for Rust versions or commit: (crates.io aws-lc-sys v0.17.0)

System information: for linux, below info can be collected by running uname -srvmp

  • CPU architecture: x86_64
  • CPU name: Unknown
  • OS: Windows

Build log:

   Compiling windows_x86_64_gnu v0.52.5
   Compiling windows_x86_64_gnu v0.48.5
   Compiling typenum v1.17.0
   Compiling generic-array v0.14.7
   Compiling ring v0.17.8
   Compiling winapi-x86_64-pc-windows-gnu v0.4.0
   Compiling futures-util v0.3.30
   Compiling der v0.7.9
   Compiling aws-lc-sys v0.17.0
   Compiling rand_chacha v0.3.1
   Compiling rustls-pki-types v1.7.0
   Compiling ff v0.13.0
   Compiling byteorder v1.5.0
   Compiling windows-targets v0.52.5
   Compiling windows-targets v0.48.5
   Compiling winapi v0.3.9
   Compiling base16ct v0.2.0
   Compiling rand v0.8.5
   Compiling windows-sys v0.52.0
   Compiling windows-sys v0.48.0
   Compiling parking_lot_core v0.9.10
   Compiling group v0.13.0
   Compiling opaque-debug v0.3.1
   Compiling mirai-annotations v1.12.0
   Compiling parking_lot v0.12.3
   Compiling syn v1.0.109
   Compiling ring v0.16.20
   Compiling unicode-normalization v0.1.23
   Compiling http v1.1.0
   Compiling signature v2.2.0
   Compiling unicode-bidi v0.3.15
   Compiling spin v0.5.2
   Compiling untrusted v0.7.1
   Compiling percent-encoding v2.3.1
   Compiling rustls v0.21.12
   Compiling ryu v1.0.18
   Compiling crossbeam-utils v0.8.20
   Compiling form_urlencoded v1.2.1
   Compiling futures-executor v0.3.30
   Compiling pin-project v1.1.5
   Compiling http v0.2.12
   Compiling rustc-hash v1.1.0
   Compiling httparse v1.8.0
   Compiling zstd-sys v1.6.2+zstd.1.5.1
   Compiling futures v0.3.30
   Compiling http-body v1.0.0
   Compiling idna v0.5.0
   Compiling fastrand v1.9.0
   Compiling ipnet v2.9.0
   Compiling try-lock v0.2.5
   Compiling rustls-webpki v0.101.7
   Compiling sct v0.7.1
   Compiling base64 v0.21.7
   Compiling atomic-waker v1.1.2
   Compiling serde_json v1.0.117
   Compiling want v0.3.1
   Compiling rustls-pemfile v1.0.4
   Compiling url v2.5.0
   Compiling socket2 v0.5.7
   Compiling schannel v0.1.23
   Compiling winapi-util v0.1.8
   Compiling num-traits v0.2.19
   Compiling spki v0.7.3
   Compiling mio v0.8.11
   Compiling quinn-udp v0.4.1
   Compiling match_cfg v0.1.0
   Compiling ipconfig v0.3.2
   Compiling arrayvec v0.7.4
   Compiling base64 v0.22.1
   Compiling powerfmt v0.2.0
   Compiling httpdate v1.0.3
   Compiling rustls v0.22.4
   Compiling tokio v1.37.0
   Compiling radium v0.7.0
   Compiling deranged v0.3.11
   Compiling quinn-proto v0.10.6
   Compiling parking_lot_core v0.8.6
   Compiling pkcs8 v0.10.2
   Compiling hostname v0.3.1
   Compiling blake3 v1.5.1
   Compiling winreg v0.50.0
   Compiling rustls-native-certs v0.6.3
   Compiling same-file v1.0.6
   Compiling idna v0.4.0
   Compiling ed25519 v2.2.3
   Compiling windows-core v0.51.1
   Compiling aho-corasick v1.1.3
   Compiling instant v0.1.13
   Compiling linked-hash-map v0.5.6
   Compiling tower-service v0.3.2
   Compiling c2rust-bitfields-derive v0.18.0
   Compiling num-conv v0.1.0
   Compiling tap v1.0.1
   Compiling widestring v1.1.0
   Compiling utf8parse v0.2.1
   Compiling regex-syntax v0.8.3
   Compiling constant_time_eq v0.3.0
The following warnings were emitted during compilation:

warning: aws-lc-sys@0.17.0: Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu

error: failed to run custom build command for `aws-lc-sys v0.17.0`

Caused by:
  process didn't exit successfully: `/target/debug/build/aws-lc-sys-9800ca95c1e3d479/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  cargo:warning=Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu
  cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
  cargo:rustc-cfg=use_bindgen_generated
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-gnu = None
  CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-pc-windows-gnu = None
  CMAKE_GENERATOR_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-pc-windows-gnu = None
  CMAKE_PREFIX_PATH_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-pc-windows-gnu = None
  CMAKE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE = None
  CMAKE = Some("cmake")
  running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_BUILD_TYPE=debug" "-DBORINGSSL_PREFIX=aws_lc_0_17_0_" "-DBORINGSSL_PREFIX_HEADERS=/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/generated-include" "-DBUILD_TESTING=OFF" "-DBUILD_LIBSSL=OFF" "-DDISABLE_PERL=ON" "-DDISABLE_GO=ON" "-DCMAKE_SYSTEM_NAME=Windows" "-DCMAKE_SYSTEM_PROCESSOR=AMD64" "-DCMAKE_INSTALL_PREFIX=/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_C_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/x86_64-w64-mingw32-g++-posix" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "--no-warn-unused-cli"
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 7.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/x86_64-w64-mingw32-gcc-posix - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Go not found. Disabling some code generation and using pre-generated code in generated-src/
  -- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
  -- stdalign_check.c probe is positive, enabling AWS_LC_STDALIGN_AVAILABLE
  -- builtin_swap_check.c probe is positive, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
  -- The ASM_NASM compiler identification is NASM
  -- Found assembler: /usr/bin/nasm
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
  -- Found Threads: TRUE
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build
  running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && MAKEFLAGS="-j --jobserver-fds=8,9 --jobserver-auth=8,9" "cmake" "--build" "." "--target" "install" "--config" "Debug"
  [  0%] Built target boringssl_prefix_symbols
  [  1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.obj
  [  1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/fips_shared_support.c.obj
  [  2%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/cpucap/cpucap.c.obj
  [  2%] Building ASM_NASM object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj
  aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/build.make:120: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj' failed
  CMakeFiles/Makefile2:257: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all' failed
  Makefile:135: recipe for target 'all' failed

  --- stderr
  Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/win-x86_64/crypto/ to /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto
  /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto/fipsmodule/aesni-gcm-avx512.asm:10: fatal: unable to open include file `openssl/boringssl_prefix_symbols_nasm.inc'
  make[2]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj] Error 1
  make[1]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all] Error 2
  make: *** [all] Error 2
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 2

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
@justsmth
Copy link
Contributor

justsmth commented May 28, 2024 •
edited
Loading

I'm unable to reproduce the error you're reporting using using the aws-lc-rs repo. I believe the problem related to the Cross.toml configuration that you have here.

For reference, you can see the configuration that we use for Cross here, which extends the configuration from our Cross Dockerfile.

It's not clear to me what a few of the steps in your configuration are attempting, but I suspect your configuration could be much simpler. Possibly something like:

[build]
pre-build = [
  "apt-get install --assume-yes --no-install-recommends build-essential cmake nasm clang",
  "cargo install --force --locked bindgen-cli"
]

EDIT:
I experimented more with this. The "cargo install --force --locked bindgen-cli" doesn't work the way I'd expect, which likely explains the "rustup" commands you were using in your Cross.toml. I'll see if I can find a better solution.

@justsmth justsmth mentioned this issue May 28, 2024
Closed
@zonyitoo
Copy link
Author

Yes, there is no cargo in the base image. So I have to install one.

The Dockerfile in this repo, installs build-essential cmake golang-go, and Cross.toml also installs nasm clang wget and also a winehq-stable.

Comparing to my own Cross.toml, I only installed build-essential cmake nasm clang, the missing one are golang-go winehq-stable. Do they necessary while building aws-lc?

@justsmth justsmth mentioned this issue May 28, 2024
Merged
@justsmth
Copy link
Contributor

justsmth commented May 28, 2024 •
edited
Loading

the missing one are golang-go winehq-stable. Do they necessary while building aws-lc?

golang-go is only needed for FIPS builds. Installing the latest winehq-stable was (and still is?) needed for testing on x86_64-pc-windows-gnu.

I put up a draft PR that follows the pattern you use for installing bindgen-cli inside the container.

@justsmth justsmth mentioned this issue May 28, 2024
Closed
@justsmth
Copy link
Contributor

I was able to get your repo to cross-build using aws-lc-rs. I posted my changes on this branch. Feel free to adapt parts of that change for your own purposes.

I also updated this PR as ready for review. Feel free to contact us if you have any other problems. Thanks!

@zonyitoo
Copy link
Author

zonyitoo commented Jun 1, 2024 •
edited
Loading

I have just merged your changed into my master branch, but still not working well.

$ cross build --target "x86_64-pc-windows-gnu" --features "full-extra"
...
The following warnings were emitted during compilation:

warning: aws-lc-sys@0.17.0: Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu

error: failed to run custom build command for `aws-lc-sys v0.17.0`

Caused by:
  process didn't exit successfully: `/target/debug/build/aws-lc-sys-9800ca95c1e3d479/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  cargo:warning=Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu
  cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
  cargo:rustc-cfg=use_bindgen_generated
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-gnu = None
  CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-pc-windows-gnu = None
  CMAKE_GENERATOR_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-pc-windows-gnu = None
  CMAKE_PREFIX_PATH_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-pc-windows-gnu = None
  CMAKE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE = None
  CMAKE = Some("cmake")
  running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_BUILD_TYPE=debug" "-DBORINGSSL_PREFIX=aws_lc_0_17_0_" "-DBORINGSSL_PREFIX_HEADERS=/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/generated-include" "-DBUILD_TESTING=OFF" "-DBUILD_LIBSSL=OFF" "-DDISABLE_PERL=ON" "-DDISABLE_GO=ON" "-DCMAKE_SYSTEM_NAME=Windows" "-DCMAKE_SYSTEM_PROCESSOR=AMD64" "-DCMAKE_INSTALL_PREFIX=/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_C_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/x86_64-w64-mingw32-g++-posix" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "--no-warn-unused-cli"
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 7.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/x86_64-w64-mingw32-gcc-posix - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Go not found. Disabling some code generation and using pre-generated code in generated-src/
  -- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
  -- stdalign_check.c probe is positive, enabling AWS_LC_STDALIGN_AVAILABLE
  -- builtin_swap_check.c probe is positive, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
  -- The ASM_NASM compiler identification is NASM
  -- Found assembler: /usr/bin/nasm
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
  -- Found Threads: TRUE
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build
  running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && MAKEFLAGS="-j --jobserver-fds=8,9 --jobserver-auth=8,9" "cmake" "--build" "." "--target" "install" "--config" "Debug"
  [  0%] Built target boringssl_prefix_symbols
  [  1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.obj
  [  1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/fips_shared_support.c.obj
  [  2%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/cpucap/cpucap.c.obj
  [  2%] Building ASM_NASM object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj
  aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/build.make:120: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj' failed
  CMakeFiles/Makefile2:257: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all' failed
  Makefile:135: recipe for target 'all' failed

  --- stderr
  Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/win-x86_64/crypto/ to /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto
  /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto/fipsmodule/aesni-gcm-avx512.asm:10: fatal: unable to open include file `openssl/boringssl_prefix_symbols_nasm.inc'
  make[2]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj] Error 1
  make[1]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all] Error 2
  make: *** [all] Error 2
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 2

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

CI should always passed. These failed targets are not included in normal CI tests.

  • x86_64-pc-windows-gnu
  • armv7-unknown-linux-musleabihf
  • armv7-unknown-linux-gnueabihf
  • arm-unknown-linux-gnueabihf

Interestingly CI tests in your PR are all passed. WHY??

@zonyitoo
Copy link
Author

zonyitoo commented Jun 1, 2024

clang should also explicitly install the latest version clang-8 like this:

diff --git a/docker/linux-cross/Dockerfile b/docker/linux-cross/Dockerfile
index d17c46cb..5f17c9a5 100644
--- a/docker/linux-cross/Dockerfile
+++ b/docker/linux-cross/Dockerfile
@@ -4,7 +4,7 @@ FROM $CROSS_BASE_IMAGE
 ARG DEBIAN_FRONTEND=noninteractive

 RUN apt-get update && \
-    apt-get install --assume-yes --no-install-recommends build-essential cmake nasm llvm-dev libclang-dev clang  && \
+    apt-get install --assume-yes --no-install-recommends build-essential cmake nasm llvm-8-dev libclang-8-dev clang-8 && \
     git config --global --add safe.directory '*' && \
     rm -rf /tmp/*

Otherwise some of the targets may failed with these errors:

The following warnings were emitted during compilation:

warning: aws-lc-sys@0.17.0: Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd

error: failed to run custom build command for `aws-lc-sys v0.17.0`

Caused by:
  process didn't exit successfully: `/target/debug/build/aws-lc-sys-9800ca95c1e3d479/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  cargo:warning=Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd
  cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES

  --- stderr
  Failure invoking external bindgen!

  bindgen-PARAMS: --prefix-link-name aws_lc_0_17_0_ --rust-target 1.59 --with-derive-default --with-derive-eq --allowlist-file .*(/|\\)openssl(/|\\)[^/\\]+\.h --allowlist-file .*(/|\\)rust_wrapper\.h --rustified-enum point_conversion_form_t --default-macro-constant-type signed --formatter rustfmt --output /target/x86_64-unknown-freebsd/debug/build/aws-lc-sys-6ed5f5aacfdd0e3e/out/bindings.rs --raw-line
  // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
  // SPDX-License-Identifier: Apache-2.0 OR ISC
   /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/include/rust_wrapper.h -- -I /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/include -I /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/include
  bindgen-STDOUT: panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.4.0/src/lib.rs:1731:1:
  `libclang` function not loaded: `clang_getTranslationUnitTargetInfo`. This crate requires that `libclang` 3.9 or later be installed on your system. For more information on how to accomplish this, see here: https://rust-lang.github.io/rust-bindgen/requirements.html#installing-clang-39

  bindgen-STDERR: /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/include/openssl/base.h:61:10: fatal error: 'stdlib.h' file not found

  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/builder/main.rs:459:5:
  aws-lc-sys build failed. Please enable the 'bindgen' feature on aws-lc-rs or aws-lc-sys.For more information, see the aws-lc-rs User Guide: https://aws.github.io/aws-lc-rs/index.html
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

@zonyitoo
Copy link
Author

zonyitoo commented Jun 1, 2024

Ok.. I know why you cannot reproduce. shadowsocks-rust could compile with default features, but fails with --features "full-extra" and --features "full".

@justsmth Please try again with --features "full" or --features "full-extra".

I am still investigating what's the key differences between the dependencies.

@zonyitoo
Copy link
Author

zonyitoo commented Jun 1, 2024 •
edited
Loading

Here is the key differences between the two dependency trees: If there are rustls 0.23.* and rustls 0.22.* (or any other older version of rustls) in the dependency tree, then compilation will definately get errors on those specific platforms.

zonyitoo added a commit to shadowsocks/shadowsocks-rust that referenced this issue Jun 1, 2024
@zonyitoo
fix: rollback rustls to v0.22
0fc35d9 
rustls v0.23 starts to use aws-lc as the default crypto implementation.
But now not all the dependencies are upgraded to the latest rustls. If
there are still any older version of rustls exists in the build process,
aws-lc may fails.

- ref aws/aws-lc-rs#428
- ref aws/aws-lc-rs#427
- ref aws/aws-lc-rs#426
@justsmth justsmth reopened this Jun 3, 2024
@justsmth
Copy link
Contributor

justsmth commented Jun 3, 2024 •
edited
Loading

clang should also explicitly install the latest version clang-8 like this:

The latest version of clang is actually clang-18: https://releases.llvm.org/download.html

The libclang version installed by a recent LTS release of Ubuntu will be newer than clang-8.

Otherwise some of the targets may failed with these errors:

This output shows that the problem is that bindgen is unable to locate libclang:

...
 `libclang` function not loaded: `clang_getTranslationUnitTargetInfo`. This crate requires that `libclang` 3.9 or later be installed on your system. For more information on how to accomplish this, see here: https://rust-lang.github.io/rust-bindgen/requirements.html#installing-clang-39
...

I also had trouble with libclang being located when it's being installed from a "pre-build" in Cross.toml (as you have it). I've had more success by using a dedicated Dockerfile to setup the required libraries as we have here.

@justsmth
Copy link
Contributor

justsmth commented Jun 3, 2024 •
edited
Loading

Oh! I just now noticed the target your compiling for here:

warning: aws-lc-sys@0.17.0: Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd

We've not tested our build for the x86_64-unknown-freebsd target, so it's possibly broken.

If this is a platform you'd like us to support, please submit a separate issue requesting support for it.

@zonyitoo
Copy link
Author

zonyitoo commented Jun 4, 2024

Ok, never mind, this was the platform I was running my build tests.

clang-8 is the latest version could be installed inside the Docker container. apt-get will install clang 3.5 by default.

On the other hand, I think the key issue here is that: If compiling with older version of rustls, aws-ls-sys will get different errors on every platforms that is not "pregenerated".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cross-compile using bindgen-cli justsmth/aws-lc-rs
2 participants
@zonyitoo @justsmth