[package] pybind11_json/0.2.13: CMakeDeps incorrectly links to pybind11_all_do_not_use

[ZbigniewRA]

Description

When CMakeDeps is used for pybind11 it generates the following in pybind11_json-Target-release.cmake (tested on Windows):

set_property(TARGET pybind11_json_DEPS_TARGET
             APPEND PROPERTY INTERFACE_LINK_LIBRARIES
             $<$<CONFIG:Release>:${pybind11_json_FRAMEWORKS_FOUND_RELEASE}>
             $<$<CONFIG:Release>:${pybind11_json_SYSTEM_LIBS_RELEASE}>
             $<$<CONFIG:Release>:pybind11_all_do_not_use;nlohmann_json::nlohmann_json>)

As the name suggests pybind11_all_do_not_use is not meant to be linked to directly. Instead pybind11::pybind11 and/or other components defined by pybind11 should be linked.

I tried to fix the recipe by adding this to package_info():

self.cpp_info.requires = ["pybind11::pybind11", "nlohmann_json::nlohmann_json"]

But it has no effect.
Martin Valgur kindly suggested using pybind11::pybind11_ could work, but the result is even more surprising:

             $<$<CONFIG:Release>:pybind11::pybind11;nlohmann_json::nlohmann_json;pybind11_all_do_not_use>)

This problem is reproducible in the test_package included with pybind11 recipe.
Please note that test_package builds and works, but pybind11_all_do_not_use includes all components contained in pybind11, and is undesirable. It includes components both for module development and python embedding, and drags in assortment of components for very specific use cases, that are not meant to be the default. Hence the do_not_use name.

Package and Environment Details

• Package Name/Version: pybind11/0.2.13
• Operating System+version: Windows 11
• Compiler+version: Visual Studio 2022
• Docker image: N/A
• Conan version: conan 2.8.1
• Python version: Python 3.12.6

Conan profile

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=194
os=Windows

Build profile:
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=194
os=Windows

Steps to reproduce

Build test_package from pybind11 recipe on Conan Center:

conan create . --version 0.2.13

Logs

Click to expand log

======== Exporting recipe to the cache ========
pybind11_json/0.2.13: Exporting package recipe: C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\conanfile.py
pybind11_json/0.2.13: exports: File 'conandata.yml' found. Exporting it...
pybind11_json/0.2.13: Copied 1 '.yml' file: conandata.yml
pybind11_json/0.2.13: Copied 1 '.py' file: conanfile.py
pybind11_json/0.2.13: Exported to cache folder: C:\Users\Zbigniew\.conan2\p\pybin9f76d1fcceede\e
pybind11_json/0.2.13: Exported: pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb (2024-10-22 06:36:12 UTC)

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=194
os=Windows

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=194
os=Windows


======== Computing dependency graph ========
Graph root
    cli
Requirements
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2 - Cache
    pybind11/2.10.4#bfa96f955a3f5d975f2ba962c40000ff - Cache
    pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb - Cache

======== Computing necessary packages ========
pybind11_json/0.2.13: Forced build from source
Requirements
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2:da39a3ee5e6b4b0d3255bfef95601890afd80709#2d1a5b1f5d673e1dab536bed20ce000b - Cache
    pybind11/2.10.4#bfa96f955a3f5d975f2ba962c40000ff:da39a3ee5e6b4b0d3255bfef95601890afd80709#123c7aa30a672ba817d5a10fb563c4b9 - Cache
    pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb:da39a3ee5e6b4b0d3255bfef95601890afd80709 - Build

======== Installing packages ========
nlohmann_json/3.11.2: Already installed! (1 of 3)
pybind11/2.10.4: Already installed! (2 of 3)
pybind11_json/0.2.13: Calling source() in C:\Users\Zbigniew\.conan2\p\pybin9f76d1fcceede\s\src
pybind11_json/0.2.13: Unzipping 0.2.13.tar.gz to .

-------- Installing package pybind11_json/0.2.13 (3 of 3) --------
pybind11_json/0.2.13: Building from source
pybind11_json/0.2.13: Package pybind11_json/0.2.13:da39a3ee5e6b4b0d3255bfef95601890afd80709
pybind11_json/0.2.13: Building your package in C:\Users\Zbigniew\.conan2\p\b\pybinc3bf9f901fb8c\b
pybind11_json/0.2.13: Generating aggregated env files
pybind11_json/0.2.13: Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat']
pybind11_json/0.2.13: Package 'da39a3ee5e6b4b0d3255bfef95601890afd80709' built
pybind11_json/0.2.13: Build folder C:\Users\Zbigniew\.conan2\p\b\pybinc3bf9f901fb8c\b\build-release
pybind11_json/0.2.13: Generating the package
pybind11_json/0.2.13: Packaging in folder C:\Users\Zbigniew\.conan2\p\b\pybinc3bf9f901fb8c\p
pybind11_json/0.2.13: Calling package()
pybind11_json/0.2.13: package(): Packaged 1 '.hpp' file: pybind11_json.hpp
pybind11_json/0.2.13: package(): Packaged 1 file: LICENSE
pybind11_json/0.2.13: Created package revision 85db3841791936b3dd8e733412e33924
pybind11_json/0.2.13: Package 'da39a3ee5e6b4b0d3255bfef95601890afd80709' created
pybind11_json/0.2.13: Full package reference: pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb:da39a3ee5e6b4b0d3255bfef95601890afd80709#85db3841791936b3dd8e733412e33924
pybind11_json/0.2.13: Package folder C:\Users\Zbigniew\.conan2\p\b\pybinc3bf9f901fb8c\p
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: pybind11/2.10.4
WARN: deprecated:     'cpp_info.build_modules' used in: pybind11/2.10.4

======== Launching test_package ========

======== Computing dependency graph ========
Graph root
    pybind11_json/0.2.13 (test package): C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\conanfile.py
Requirements
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2 - Cache
    pybind11/2.10.4#bfa96f955a3f5d975f2ba962c40000ff - Cache
    pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb - Cache

======== Computing necessary packages ========
Requirements
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2:da39a3ee5e6b4b0d3255bfef95601890afd80709#2d1a5b1f5d673e1dab536bed20ce000b - Cache
    pybind11/2.10.4#bfa96f955a3f5d975f2ba962c40000ff:da39a3ee5e6b4b0d3255bfef95601890afd80709#123c7aa30a672ba817d5a10fb563c4b9 - Cache
    pybind11_json/0.2.13#56dae1dde7fcf52ab8dac5cf18236deb:da39a3ee5e6b4b0d3255bfef95601890afd80709#85db3841791936b3dd8e733412e33924 - Cache

======== Installing packages ========
nlohmann_json/3.11.2: Already installed! (1 of 3)
pybind11/2.10.4: Already installed! (2 of 3)
pybind11_json/0.2.13: Already installed! (3 of 3)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: pybind11/2.10.4
WARN: deprecated:     'cpp_info.build_modules' used in: pybind11/2.10.4

======== Testing the package ========
Removing previously existing 'test_package' build folder: C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release
pybind11_json/0.2.13 (test package): Test package build: build\msvc-194-x86_64-20-release
pybind11_json/0.2.13 (test package): Test package build folder: C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release
pybind11_json/0.2.13 (test package): Writing generators to C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release\generators
pybind11_json/0.2.13 (test package): Generator 'CMakeDeps' calling 'generate()'
pybind11_json/0.2.13 (test package): CMakeDeps necessary find_package() and targets for your CMakeLists.txt
    find_package(pybind11_json)
    target_link_libraries(... pybind11_json::pybind11_json)
pybind11_json/0.2.13 (test package): Generator 'CMakeToolchain' calling 'generate()'
pybind11_json/0.2.13 (test package): CMakeToolchain generated: conan_toolchain.cmake
pybind11_json/0.2.13 (test package): CMakeToolchain generated: C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release\generators\CMakePresets.json
pybind11_json/0.2.13 (test package): CMakeToolchain generated: C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\CMakeUserPresets.json
pybind11_json/0.2.13 (test package): Generating aggregated env files
pybind11_json/0.2.13 (test package): Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat']

======== Testing the package: Building ========
pybind11_json/0.2.13 (test package): Calling build()
pybind11_json/0.2.13 (test package): Running CMake.configure()
pybind11_json/0.2.13 (test package): RUN: cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package"
-- Using Conan toolchain: C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package/build/msvc-194-x86_64-20-release/generators/conan_toolchain.cmake
-- Conan toolchain: CMAKE_GENERATOR_TOOLSET=v143
-- Conan toolchain: Setting CMAKE_MSVC_RUNTIME_LIBRARY=$<$<CONFIG:Release>:MultiThreadedDLL>
-- Conan toolchain: C++ Standard 20 with extensions OFF
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631.
-- The CXX compiler identification is MSVC 19.41.34120.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Warning: Standard CMAKE_CXX_STANDARD value defined in conan_toolchain.cmake to 20 has been modified to 11 by C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package/CMakeLists.txt
-- Conan: Component target declared 'pybind11_json::pybind11_json'
-- Conan: Target declared 'nlohmann_json::nlohmann_json'
-- Conan: Component target declared 'pybind11::headers'
-- Conan: Component target declared 'pybind11::pybind11'
-- Conan: Component target declared 'pybind11::embed'
-- Conan: Component target declared 'pybind11::module'
-- Conan: Component target declared 'pybind11::python_link_helper'
-- Conan: Component target declared 'pybind11::windows_extras'
-- Conan: Component target declared 'pybind11::lto'
-- Conan: Component target declared 'pybind11::thin_lto'
-- Conan: Component target declared 'pybind11::opt_size'
-- Conan: Component target declared 'pybind11::python2_no_register'
-- Conan: Target declared 'pybind11_all_do_not_use'
-- Conan: Including build module from 'C:/Users/Zbigniew/.conan2/p/pybinf0d1019912839/p/lib/cmake/pybind11/pybind11Common.cmake'
CMake Warning (dev) at C:/Users/Zbigniew/.conan2/p/pybinf0d1019912839/p/lib/cmake/pybind11/FindPythonLibsNew.cmake:98 (find_package):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

Call Stack (most recent call first):
  C:/Users/Zbigniew/.conan2/p/pybinf0d1019912839/p/lib/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
  C:/Users/Zbigniew/.conan2/p/pybinf0d1019912839/p/lib/cmake/pybind11/pybind11Common.cmake:180 (include)
  build/msvc-194-x86_64-20-release/generators/pybind11-config.cmake:38 (include)
  C:/Program Files/Microsoft Visual Studio/2022/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.29/Modules/CMakeFindDependencyMacro.cmake:76 (find_package)
  build/msvc-194-x86_64-20-release/generators/pybind11_json-config.cmake:24 (find_dependency)
  CMakeLists.txt:6 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found PythonInterp: C:/Code/sde-conan-packages-env/Scripts/python.exe (found suitable version "3.12.6", minimum required is "3.6")
-- Found PythonLibs: C:/Python312/libs/python312.lib
-- Performing Test HAS_MSVC_GL_LTCG
-- Performing Test HAS_MSVC_GL_LTCG - Success
-- Configuring done (2.7s)
-- Generating done (0.0s)
-- Build files have been written to: C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package/build/msvc-194-x86_64-20-release

pybind11_json/0.2.13 (test package): Running CMake.build()
pybind11_json/0.2.13 (test package): RUN: cmake --build "C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release" --config Release
MSBuild version 17.11.9+a69bbaaf5 for .NET Framework

  1>Checking Build System
  Building Custom Rule C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package/CMakeLists.txt
  test_package.cpp
  Generating code
  Finished generating code
  test_package.vcxproj -> C:\Code\ThirdParty\conan-center-index\recipes\pybind11_json\all\test_package\build\msvc-194-x86_64-20-release\Release\test_package.exe
  Building Custom Rule C:/Code/ThirdParty/conan-center-index/recipes/pybind11_json/all/test_package/CMakeLists.txt


======== Testing the package: Executing test ========
pybind11_json/0.2.13 (test package): Running test()
pybind11_json/0.2.13 (test package): RUN: Release\test_package
Converted nlohmann::json contents: [{"hello":"world","number":1234}]
Converted py::dict contents: {hello:world, number:1234}


Process finished with exit code 0

[SpaceIm]

If I add self.cpp_info.requires = ["pybind11::pybind11_", "nlohmann_json::nlohmann_json"] to package_info() of pybind11_json recipe, I see $<$<CONFIG:Release>:pybind11::pybind11;nlohmann_json::nlohmann_json> in pybind11_json-Target-release.cmake.

So it's the way to go (but I don't know if pybind11::pybind11_ is the correct dependency component of pybind11 for pybind_json, maybe it's another one).

[ZbigniewRA]

Yes, that works.
This will technically be a breaking change, since test_package requires now linking to pybind11::embed. But this is how it should be.

So the remaining issues are:

• pybdin11_LIBRARIES is still set to pybind11_all_do_not_use.
• We're not sure if pybind11::pybind11_ is the correct dependency component of pybind11 for pybind_json, maybe it's another one.

I think both issues are non critical.

• pybdin11_LIBRARIES can just be avoided. There is pybind11::pybind11 target that can be used instead. Also that is an issue with pybind11 package, not this one.
• Alternative for pybind11::pybind11_ would be pybind11::headers. pybind11::pybind11_ pulls in Python headers. Both should work fine.

[ZbigniewRA]

I think I'll need some help with the PR that fixes this issue: #25703

• Conan 1 build is marked as failed, but the logs show that test succeeded. So why did the build fail?
• Conan 2 build fails because Can't find a 'cpython/3.12.7' package binary. Is there a way to make Conan CI build this package?