[bug] conan 2.x: linking against a shared lib with private shared dependencies fails on Linux

[SpaceIm]

Environment details

• Operating System+version: Linux agent in c3i
• Compiler+version: gcc any version
• Conan version: conan 2.0.0-beta8 I guess
• Python version: I don't know

Steps to reproduce

On Linux, run conan create *:shared=True against any recipe with private dependencies, and see link errors in test package:

conan create . libpng/1.6.39@ -o *:shared=True

Logs

********************************************************************************
conan test conan-center-index/recipes/libpng/all/test_package/conanfile.py libpng/1.6.38@#00bf8a010936677ddb13cabdaa82dcce -pr:h /home/conan/w/prod-v2/BuildSingleReference/17875/5d252ca9-86d9-4830-a9f0-ecb330766f27/profile_linux_11_libstdcpp11_gcc_release_64.-shared-True.txt -c:h tools.system.package_manager:mode=install -c:h tools.system.package_manager:sudo=True -pr:b /home/conan/w/prod-v2/BuildSingleReference/17875/5d252ca9-86d9-4830-a9f0-ecb330766f27/profile_linux_11_libstdcpp11_gcc_release_64.-shared-True.txt -c:b tools.system.package_manager:mode=install -c:b tools.system.package_manager:sudo=True
********************************************************************************

-------- Input profiles --------
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[options]
*/*:shared=True
[conf]
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[options]
*/*:shared=True
[conf]
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True


-------- test_package: Computing dependency graph --------
Graph root
    libpng/1.6.38 (test package): /home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/conanfile.py
Requirements
    libpng/1.6.38#00bf8a010936677ddb13cabdaa82dcce - Cache
    zlib/1.2.13#13c96f538b52e1600c40b88994de240f - Cache

-------- test_package: Computing necessary packages --------

-------- Computing necessary packages --------
Requirements
    libpng/1.6.38#00bf8a010936677ddb13cabdaa82dcce:e143b5ce0c0cc3fb416c452b81064dd1ae9d6a18#df554e7544c217c0b7f1ba118300d050 - Cache
    zlib/1.2.13#13c96f538b52e1600c40b88994de240f:abe5e2b04ea92ce2ee91bc9834317dbe66628206#441a647ceea3b33b1b0dbe1bef7a807d - Cache

-------- test_package: Installing packages --------

-------- Installing (downloading, building) binaries... --------
zlib/1.2.13: Already installed!
WARN: The use of 'cpp_info.names' is deprecated in Conan 2.0 and will be removed in Conan 2.X. Please, update your recipes unless you are maintaining compatibility with Conan 1.X
WARN: The use of 'cpp_info.names' is deprecated in Conan 2.0 and will be removed in Conan 2.X. Please, update your recipes unless you are maintaining compatibility with Conan 1.X
libpng/1.6.38: Already installed!
WARN: The use of 'cpp_info.names' is deprecated in Conan 2.0 and will be removed in Conan 2.X. Please, update your recipes unless you are maintaining compatibility with Conan 1.X
WARN: The use of 'cpp_info.names' is deprecated in Conan 2.0 and will be removed in Conan 2.X. Please, update your recipes unless you are maintaining compatibility with Conan 1.X

-------- Testing the package --------
libpng/1.6.38 (test package): Writing generators to /home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release/generators
libpng/1.6.38 (test package): Generator 'CMakeDeps' calling 'generate()'
libpng/1.6.38 (test package): Generator 'CMakeToolchain' calling 'generate()'
libpng/1.6.38 (test package): Preset 'release' added to CMakePresets.json. Invoke it manually using 'cmake --preset release'
libpng/1.6.38 (test package): If your CMake version is not compatible with CMakePresets (<3.19) call cmake like: 'cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release'
libpng/1.6.38 (test package): Generator 'VirtualRunEnv' calling 'generate()'
libpng/1.6.38 (test package): Aggregating env generators

-------- Testing the package: Building --------
libpng/1.6.38 (test package): Calling build()
libpng/1.6.38 (test package): CMake command: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/."
libpng/1.6.38 (test package): RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/."
-- Using Conan toolchain: /home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release/generators/conan_toolchain.cmake
-- The C compiler identification is GNU 11.1.0
-- Check for working C compiler: /usr/local/bin/cc
-- Check for working C compiler: /usr/local/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Conan: Target declared 'PNG::PNG'
-- Conan: Target declared 'ZLIB::ZLIB'
-- Configuring done
-- Generating done
-- Build files have been written to: /home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release
libpng/1.6.38 (test package): CMake command: cmake --build "/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release" -- -j3
libpng/1.6.38 (test package): RUN: cmake --build "/home/conan/w/prod-v2/BuildSingleReference/conan-center-index/recipes/libpng/all/test_package/test_output/build/Release" -- -j3
Scanning dependencies of target test_package
[ 50%] Building C object CMakeFiles/test_package.dir/test_package.c.o
[100%] Linking C executable test_package
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/240bbd6dbee90c98/p/lib/libpng.so: undefined reference to `inflateValidate@ZLIB_1.2.9'
collect2: error: ld returned 1 exit status
CMakeFiles/test_package.dir/build.make:84: recipe for target 'test_package' failed
make[2]: *** [test_package] Error 1
CMakeFiles/Makefile2:75: recipe for target 'CMakeFiles/test_package.dir/all' failed
make[1]: *** [CMakeFiles/test_package.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
ERROR: libpng/1.6.38 (test package): Error in build() method, line 21
	cmake.build()
	ConanException: Error 2 while executing

As explained in conan-io/conan-center-index#15538 (comment), I think something is missing in CMakeDeps (and I guess all Deps like generators) so that CMake can understand that -rpath-link must be injected with cpp_info.libdirs of private transitive libs.
This issue is hidden in conan v1 because everything is public.

[memsharded]

Hi @SpaceIm

This seems to be mixing versions.

I am trying:

conan create . --version=1.6.39 --build=missing -o *:shared=True

In Linux, with Conan 2.0 (2.0.0-develop2 branch), and it works fine.

The command syntax you report above conan create . libpng/1.6.39@ is 1.X, but the report is for 2.0?
I guess there might be something different in the logs, because it is testing a different thing, with some different profiles.

[SpaceIm]

I don't know conan v2 commands so yes it's probably not exactly the command I've mentioned.
It's just the report of conan v2 pipeline in c3i for libpng: conan-io/conan-center-index#15538 (comment)
I don't understand why test package fails for */*:shared=True on Linux.

[SpaceIm]

In the meantime, since libz.so is quite common in system path, you may not be able to reproduce this issue locally.

[memsharded]

In the meantime, since libz.so is quite common in system path, you may not be able to reproduce this issue locally.

I am using conanio/gcc6 docker image to test it, if that makes it pass here locally, it should also work in c3i, same base images?

Maybe @uilianries or @prince-chrismc can have a look to the failure in C3i

[SpaceIm]

Here is another example where */*:shared fails in test package but not "all static":
ixwebsocket (zlib & mbedtls are direct private dependencies, zlib also private dependency of mbedtls):
conan-io/conan-center-index#15511 (comment)
https://c3i.jfrog.io/c3i/misc-v2/logs/pr/15511/3-linux-gcc/ixwebsocket/11.4.3//b31edb4db42e819cfc8e914dd5e31a2bae3404a4-test.txt

[100%] Linking CXX executable test_package
/opt/conan/binutils/bin/ld: warning: libmbedtls.so.13, needed by /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so, not found (try using -rpath or -rpath-link)
/opt/conan/binutils/bin/ld: warning: libmbedx509.so.1, needed by /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so, not found (try using -rpath or -rpath-link)
/opt/conan/binutils/bin/ld: warning: libmbedcrypto.so.6, needed by /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so, not found (try using -rpath or -rpath-link)
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_x509_crt_parse_file'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_pk_parse_keyfile'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_net_recv'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ctr_drbg_seed'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_strerror'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_set_hostname'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_config_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_handshake'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_conf_ca_chain'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_entropy_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_x509_crt_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ctr_drbg_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_free'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_config_free'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_setup'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_config_defaults'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_conf_authmode'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ctr_drbg_random'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_x509_crt_parse'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_conf_own_cert'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ctr_drbg_free'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_set_bio'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_net_send'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_get_verify_result'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_entropy_free'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_pk_init'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_read'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_x509_crt_free'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_entropy_func'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_conf_rng'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_ssl_write'
/opt/conan/binutils/bin/ld: /home/conan/w/prod-v2/BuildSingleReference/p/763f4d62646271c6/p/lib/libixwebsocket.so: undefined reference to `mbedtls_x509_crt_verify_info'

[czoido]

Closed by #13010, thanks for reporting!

[SpaceIm]

Thanks. I'll take a look when I have time (for example by comparing with a canonical CMake config file generated by configure_package_config_file()).

Are you sure that AutotoolsDeps & MesonDeps (maybe also PkgConfigDeps?) don't need a fix as well?

[czoido]

Thanks @SpaceIm for the suggestion, I'll open an issue so we can have some time to study those cases.