Relocation error when linking application on Ubuntu 20.04 (focal) arm64 related to libcrypto.a

[asfg84]

Describe the bug

I successfully built the sdk (version 1.11.342) on Ubuntu 20.04 (focal) and 22.04 (jammy) for amd64, arm64 and armhf.

When I link against the SDK, it works for all architectures and Ubuntu versions except for focal arm64. There, linking fails with relocation errors related to libcrypto.a.

Expected Behavior

Linking should work without errors on focal and jammy on all architectures.

Current Behavior

When I link against the SDK, it works for all Ubuntu versions and architectures except for focal arm64. There, linking fails with

[100%] Linking CXX shared library libs3_client.so
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha1-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
195/usr/lib/aarch64-linux-gnu/libcrypto.a(sha1-armv8.o): in function `sha1_block_armv8':
196(.text+0x1240): dangerous relocation: unsupported relocation
197/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(chacha-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
198/usr/lib/aarch64-linux-gnu/libcrypto.a(chacha-armv8.o):(.text+0x20): dangerous relocation: unsupported relocation
199/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_LO21 against symbol `poly1305_blocks' which may bind externally can not be used when making a shared object; recompile with -fPIC
200/usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): in function `poly1305_init':
201(.text+0x40): dangerous relocation: unsupported relocation
202/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_LO21 against symbol `poly1305_emit' which may bind externally can not be used when making a shared object; recompile with -fPIC
203(.text+0x48): dangerous relocation: unsupported relocation
204/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
205/usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): in function `poly1305_emit_neon':
206(.text+0x9a0): dangerous relocation: unsupported relocation
207/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha256-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
208/usr/lib/aarch64-linux-gnu/libcrypto.a(sha256-armv8.o): in function `sha256_block_data_order':
209(.text+0xf88): dangerous relocation: unsupported relocation
210/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha512-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
211/usr/lib/aarch64-linux-gnu/libcrypto.a(sha512-armv8.o): in function `sha512_block_data_order':
212(.text+0x1108): dangerous relocation: unsupported relocation
213collect2: error: ld returned 1 exit status

Reproduction Steps

Build the SDK on focal arm64 using:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="transfer" -DBUILD_SHARED_LIBS=OFF -DMINIMIZE_SIZE=ON -DENABLE_TESTING=OFF -DENABLE_ZLIB_REQUEST_COMPRESSION=OFF ..

Create an application which creates an S3Client, and link against the SDK library:

set(SOURCES
  s3_client.cc)

add_library(s3_client SHARED ${SOURCES})

target_include_directories(s3_client
  PUBLIC
    ${AWSSDK_INCLUDE_DIR}
    # $<INSTALL_INTERFACE:include>
    $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include/s3_client>)

target_link_libraries(s3_client
  PUBLIC
    ${AWSSDK_LINK_LIBRARIES})

Possible Solution

No response

Additional Information/Context

I installed:

apt-get install -y curl libcurl4-openssl-dev libssl-dev uuid-dev zlib1g-dev

During building the SDK, I get the following output regarding libcrypto:

-- Found crypto: /usr/lib/aarch64-linux-gnu/libcrypto.a  
317-- LibCrypto Include Dir: /usr/include
318-- LibCrypto Shared Lib:  /usr/lib/aarch64-linux-gnu/libcrypto.so
319-- LibCrypto Static Lib:  /usr/lib/aarch64-linux-gnu/libcrypto.a
320-- Using libcrypto from the cmake path

I also tried using -DAWS_USE_CRYPTO_SHARED_LIBS=ON when building the SDK, but no change.

On all focal architectures, the output of openssl version is

OpenSSL 1.1.1f  31 Mar 2020

So I cannot see why linking succeeds on focal amd64 and armhf, but fails on arm64.

AWS CPP SDK version used

1.11.342

Compiler and Version used

gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

Operating System and version

Ubuntu 20.04.6 LTS

[DmitriyMusatkin]

This is likely due to this bug in openssl openssl/openssl#10842.

There are some workarounds in the that ticket, but looks like the issue is only fixed in 1.1.1i.

Alternative workaround might be to compile sdk against aws-lc, which should not have this issue

[asfg84]

Thank you for your response. Application linking is not an issue with older versions of the SDK, e.g. we successfully used it with lib-aws-cpp-sdk version 1.6.53. So I assume that this old version linked against the shared library of libcrypto? Is there any way I can still enforce linking against the shared libcrypto in the current version? I thought the cmake option AWS_USE_CRYPTO_SHARED_LIBS=ON would do exactly that, but I still get the same relocation errors.

[DmitriyMusatkin]

Yes, AWS_USE_CRYPTO_SHARED_LIBS=ON should work as a workaround in this case and looks like it does not an its trying to link against static crypto?

We'll need to see if we can repro it.

We recently did a pretty big refactoring of our crypto usage and im wondering if there is a issue with that somewhere. Would you mind trying a release that is slightly older and see if it still occurs (lets say around 1.11.300)?

[asfg84]

Thanks for the suggestion. I built the SDK version 1.11.300 with and without setting AWS_USE_CRYPTO_SHARED_LIBS=ON but I get the same behavior as for the most recent version in both cases.

[asfg84]

I found that I need to go back to version 1.9.234 to not get the relocation errors. In 1.9.235 the relocation errors start on arm64.

[asfg84]

After spending some more time on this, I can confirm that building the SDK with USE_OPENSSL=OFF (building with aws-lc) solves the problem. However, the SDK can then not be installed on systems that already have openssl installed, so this is not an option for us.
I came to the following solution: There is an undocumented cmake parameter called BUILD_OPENSSL which builds version 1.1.1g of openssl, where the above mentioned error is fixed (openssl/openssl#10842), and also statically links this version. So I use this cmake parameter for the focal builds:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="transfer" -DBUILD_SHARED_LIBS=OFF -DMINIMIZE_SIZE=ON -DENABLE_TESTING=OFF -DENABLE_ZLIB_REQUEST_COMPRESSION=OFF -DUSE_OPENSSL=ON -DBUILD_OPENSSL=ON ..

Then, linking the SDK in my application works.

[sbiscigl]

speaking on AWS_USE_CRYPTO_SHARED_LIBS=ON you also need to specify S2N_USE_CRYPTO_SHARED_LIBS=ON which lives in the S2N project. if you can give that a shot let me know if it works, if it does we can add missing documentation for it in our cmake parameters

[asfg84]

Thanks, I also tried it with -DAWS_USE_CRYPTO_SHARED_LIBS=ON -DS2N_USE_CRYPTO_SHARED_LIBS=ON but same result, I still had the relocation errors when linking the application on focal arm64.

[jmklix]

@asfg84 did you have any other questions/problems with the crypto library that is used by this sdk?

[asfg84]

@jmklix no thanks, I could build and use the sdk with the above mentioned cmake options:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="transfer" -DBUILD_SHARED_LIBS=OFF -DMINIMIZE_SIZE=ON -DENABLE_TESTING=OFF -DENABLE_ZLIB_REQUEST_COMPRESSION=OFF -DUSE_OPENSSL=ON -DBUILD_OPENSSL=ON ..

Would be helpful to also document the BUILD_OPENSSL cmake option, though

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.