[question] generator for modern cmake

[boussaffawalid]

Conan supports many cmake generators:

• cmake
• cmake_multi
• cmake_paths
• cmake_find_package
• cmake_find_package_multi
• CMakeDeps

Currently we use cmake and cmake_multi but they have a huge limitation (no components support). So I'm thinking to switch to cmake_find_package and cmake_find_package_multi but it seems that components support is also experimental. And then there is CMakeDeps which looks like a reimplementation of cmake_find_package

My understanding is that these different generators were created to simplify conan integration with existing cmake projects! But What is the most recommended generator to use if we do conan oriented modern cmake ?

[memsharded]

CMakeDeps is the only one that will survive in Conan 2.0. It is more flexible and evolved than cmake_find_package ones, and our development efforts are focused on this one, the others are pure maintenance mode, only for serious bugs will be changed.
To be used with the new CMakeToolchain and CMake from conan.tools.cmake. They will also work nice with the new layout() feature as well.

[boussaffawalid]

Thanks for the clarification! But how does CMakeDeps works with multi config environments.

Currently we use cmake for single config IDE (like qtcreator) and cmake_multi for multi config IDE (like visual studio).

[memsharded]

But how does CMakeDeps works with multi config environments.

CMakeDeps is a multi-config generator, it will work in multi-config IDEs like VS, and single-config one (as the single-config is basically a simplification of the multi-config one, where only 1 config is installed)

[db4]

@memsharded

CMakeDeps is the only one that will survive in Conan 2.0

What about conan-cmake, can CMakeDeps be the only generator for it? And how to do something like conan_basic_setup with CMakeDeps generator?

[memsharded]

CMakeDeps is only about dependencies, and it can work with conan-cmake, (using the find_package() syntax). It will not require any call to conan_basic_setup(). The functionality of conan_basic_setup() will be superseded by the CMakeToolchain, that converts Conan settings to CMake syntax. As conan-cmake takes CMake config and convert to Conan settings, this would be unnecessary, and the CMakeToolchain would be redundant (only used in the cache when conan create).

[db4]

As conan-cmake takes CMake config and convert to Conan settings, this would be unnecessary, and the CMakeToolchain would be redundant (only used in the cache when conan create).

Well, I need conan_output_dirs_setup() that was created before by cmake generator. What should be used instead now?

[memsharded]

Conan will no longer change the CMake or project layout, and it will not provide this method. Instead it will provide the layout() method to define the project layout, which will fully automate editable packages too. This is already released as experimental in 1.37.

If you want to change the CMake behavior you can define it yourself, implementation is straightforward, actually not related to Conan, just pure convention:

set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/bin)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELWITHDEBINFO ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_MINSIZEREL ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})

Instead of changing the CMake output directories, there are several, more idiomatic options that don't require to change the output directories:

• Use CMake install feature (the CMake.install() helper can help here too).
• Customize your package() method to take into account varying output directories (a bit more effort but doable). Previous alternative seems better.
• Use the new layout() feature to define project layout. Will probably be the recommended way soon. Then use cmake.install() or the new LayoutPackager() that uses the information already defined in the layout() method.

[db4]

@memsharded Thanks a lot for the info, but I'm still struggling with a consumer recipe that uses conan-cmake/CMakeDeps. As a first step, I try to attach Boost package; files Boost-debug-x86_64-data.cmake BoostConfig.cmake BoostConfigVersion.cmake BoostTarget-debug.cmake BoostTargets.cmake are generated, but CMake cannot find them (to be sure that correct files are used I've removed standard FindBoost.cmake from CMake distribution):

if(NOT EXISTS "${CMAKE_BINARY_DIR}/conan.cmake")
   message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
   file(DOWNLOAD "https://raw.githubusercontent.com/conan-io/cmake-conan/v0.16.1/conan.cmake"
                 "${CMAKE_BINARY_DIR}/conan.cmake")
endif()

include(${CMAKE_BINARY_DIR}/conan.cmake)

conan_cmake_configure(REQUIRES boost/1.75.0
                  GENERATORS CMakeDeps)
conan_cmake_autodetect(settings)
conan_cmake_install(PATH_OR_REFERENCE .
                  BUILD missing
                  SETTINGS ${settings})
conan_load_buildinfo()

find_package(Boost)

[cmake] CMake Warning at CMakeLists.txt:81 (find_package):
[cmake]   By not providing "FindBoost.cmake" in CMAKE_MODULE_PATH this project has
[cmake]   asked CMake to find a package configuration file provided by "Boost", but
[cmake]   CMake did not find one.
[cmake] 
[cmake]   Could not find a package configuration file provided by "Boost" with any of
[cmake]   the following names:
[cmake] 
[cmake]     BoostConfig.cmake
[cmake]     boost-config.cmake
[cmake] 
[cmake]   Add the installation prefix of "Boost" to CMAKE_PREFIX_PATH or set
[cmake]   "Boost_DIR" to a directory containing one of the above files.  If "Boost"
[cmake]   provides a separate development package or SDK, be sure it has been
[cmake]   installed.

I've tried to set(CMAKE_MODULE_PATH ${CMAKE_BINARY_DIR} ${CMAKE_MODULE_PATH}), but it didn't help. I must be missing something obvious. Maybe you have an example consumer project for conan-cmake/CMakeDeps combination?

[memsharded]

You need to:

• Set also CMAKE_PREFIX_PATH
• Do not call conan_load_buildinfo() (that loads the old conanbuildinfo.cmake, no longer generated)

This works on my side:

cmake_minimum_required(VERSION 3.15)
project(MyApp CXX)

if(NOT EXISTS "${CMAKE_BINARY_DIR}/conan.cmake")
   message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
   file(DOWNLOAD "https://raw.githubusercontent.com/conan-io/cmake-conan/v0.16.1/conan.cmake"
                 "${CMAKE_BINARY_DIR}/conan.cmake")
endif()

include(${CMAKE_BINARY_DIR}/conan.cmake)

conan_cmake_configure(REQUIRES boost/1.75.0
                  GENERATORS CMakeDeps)
conan_cmake_autodetect(settings)
conan_cmake_install(PATH_OR_REFERENCE .
                  BUILD missing
                  SETTINGS ${settings})
# conan_load_buildinfo()

set(CMAKE_MODULE_PATH ${CMAKE_BINARY_DIR} ${CMAKE_MODULE_PATH})
set(CMAKE_PREFIX_PATH ${CMAKE_BINARY_DIR} ${CMAKE_PREFIX_PATH})

find_package(Boost REQUIRED)

[db4]

@memsharded Thanks a lot again. BTW, there is another problem with cmake-conan: for multi-configuration generators its docs propose

foreach(TYPE ${CMAKE_CONFIGURATION_TYPES})
    conan_cmake_autodetect(settings BUILD_TYPE ${TYPE})
    ...
endforeach()

but actually conan_cmake_autodetect does not propagate BUILD_TYPE to _conan_detect_build_type and cmake fails with

[cmake] CMake Error at build/Visual Studio Professional 2017 Release - amd64/Debug/conan.cmake:71 (message):
[cmake]   Please specify in command line CMAKE_BUILD_TYPE
[cmake]   (-DCMAKE_BUILD_TYPE=Release)

Probably this should be reported as a cmake-conan issue?

[memsharded]

Probably this should be reported as a cmake-conan issue?

Yes, please, report this in cmake-conan.

[db4]

One more question if you don't mind. Suppose I have two header-only conan packages: PkgA and PkgB, and PkgA depends on PkgB. With cmake generator I just did target_link_libraries(my_target PRIVATE CONAN_PKG::PkgA) and my_target sources could use headers from both PkgA and PkgB. With CMakeDeps this doesn't seem to work anymore: target_link_libraries(my_target PRIVATE PkgA) cannot find PkgB headers. I must be missing something again?

[db4]

Yes, please, report this in cmake-conan.

Already reported by someone else: conan-io/cmake-conan#336

[memsharded]

With CMakeDeps this doesn't seem to work anymore: target_link_libraries(my_target PRIVATE PkgA) cannot find PkgB headers. I must be missing something again?

CMakeDeps are also transitive, so what you describe should work. Maybe is the target name, and should be something like PkgA::PkgA? That might need a reproducible case, the output logs, etc., to investigate what could be happening.

[db4]

Just verified it: target_link_libraries(my_target PRIVATE PkgA::PkgA PkgB::PkgB) works, target_link_libraries(my_target PRIVATE PkgA::PkgA) doesn't. I'll try to create a minimal reproducible test case.
Sorry, it was my fault. Actually, it works as expected.

[Nipheris]

Hi @memsharded ,

Instead of changing the CMake output directories, there are several, more idiomatic options that don't require to change the output directories

By the way, what is your recommended approach to configure bindirs/libdirs in layout() method in the absense of logic like conan_output_dirs_setup()? For example, we tried to use this one: self.cpp.build.components["foobar"].libdirs = ["src/foobar"] to point to the corresponding directory in the build tree, but, surprise!, it depends on CMake generator used. Multi-configuration CMake generators add a subdirectory for the specific configuration, like src/foobar/Debug or src/foobar/Release.

Is it possible to make layout() logic independent of CMake generator used? Or reconfiguring CMake output directories is the only viable option?

[memsharded]

Hi @Nipheris

It is not possible to make layout() agnostic of the CMake generator used, because that generator is exactly the build system to be used, and every build system has a very different layout. We had to factor this in our own cmake_layout() implementation in https://docs.conan.io/en/latest/reference/conanfile/tools/layout.html.

The general idea is try to use the native, natural layout that every build system would use, and let the layout() encode that information for Conan to be used.

[Nipheris]

@memsharded Bingo! cmake_layout() looks like the missing part of the puzzle). Thanks a lot!
But this is just a useful default for simple projects, because it doesn't deal with components, for example. Anyway, we will use it as the template to define our own layout() with components in mind.

I think we need some utility functions like get_generator or is_multi_configuration somewhere in conan.tools.cmake.*, because this logic will go out of sync with CMakeToolchain very soon if implemented in the every recipe. I mean all the new features like new msvc compiler id or Ninja Multiconfig generator will break it some day.

[memsharded]

I think we need some utility functions like get_generator or is_multi_configuration somewhere in conan.tools.cmake.*, because this logic will go out of sync with CMakeToolchain very soon if implemented in the every recipe. I mean all the new features like new msvc compiler id or Ninja Multiconfig generator will break it some day.

Yes, the idea is to mature first the main helpers like CMakeToolchain, etc., and as we internally factorize the utility functions, we will make them available, but better wait until things stabilize a bit more before publishing them.

[boussaffawalid]

@memsharded When using conan new with --template=cmake_lib, I noticed that it uses CMakeToolchain.

Does that mean CMakeToolchain is preferred over CMakeDeps or it is just an arbitrary choice ?

[memsharded]

Hi @boussaffawalid

It is not one or the other, actually both of them should be used for most common cases. Please read carefully the docs in: https://docs.conan.io/en/latest/reference/conanfile/tools/cmake.html The idea is that:

• CMakeToolchain is a generator to map from Conan settings of the current package to a conan_toolchain.cmake file, to be used with -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake
• CMakeDEps is a generator to map dependencies to xxxx-config.cmake files, one per dependency, to be used with find_package()

[sully7]

This is starting to make sense to me... but running into a question regarding CMakeDeps.

In our setup, we are hoping to let conan setup, configure, and run cmake (aka, we are not yet using cmake-conan)

Let's say we have the following in our conanfile.py

requires = [
  poco/1.9.4,
  openssl/1.0.2u,
  zlib/1.2.11@otheruser/alpha
]

def generate(self):
  tc = CMakeToolchain(self)
  # This writes the "conan_toolchain.cmake"
  tc.generate()

  deps = CMakeDeps(self)
  # This writes all the config files (xxx-config.cmake)
  deps.generate()

def build(self):
  cmake = CMake(self)
  cmake.configure()
  cmake.build()

We run a conan install ., upon which:

1. CMakeToolchain generates conan_toolchain.cmake, and
2. CMakeDeps generates all the package-specific xxxx-config.cmake files.

The conan_toolchain.cmake file is automatically added into the build environment with cmake.build() - as it is running: -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake.... but what about the dependencies?

Does this mean that we need to manually add a find_package into our cmake file for every required package we have defined in our conanfile.py? All the talk in this current thread, as well as this and this seem to point that way?

For MSBuildDeps... we get a really nice and simple conandeps.props file that adds ALL the dependencies to the project with one quick add.... do we have something similar for CMake? Since all the other interfaces (such as CONAN_LIBS) were removed, is there another macro or foreach we can use that provides the names of all packages??

[memsharded]

Hi @sully7

Yes, it is necessary to add find_package() calls in the CMakeLists.txt. This is exactly what that majority of users were pushing to have, a "transparent" CMake integration via find_package(). Do you mean that you would like a conandeps.cmake file that adds all the find_package() calls, so you can just included that file?
In the worst case, this is something easily achievable with something like:

def generate(self):
     finds = ["find_package({} CONFIG REQUIRED)".format(d.ref.name) for d in self.dependencies.direct_host.values()]
     save(self, "conandeps.cmake", "\n".join(finds)

[sully7]

@memsharded - Thanks so much for this!!

For my use case, we had to add a second line to it to also include the target_link_libraries. Not a python expert by any means... but this worked for us!

   deps_packages = []
   for d in self.dependencies.direct_host.values():
     deps_packages.append("find_package({} CONFIG REQUIRED)".format(d.ref.name))
     deps_packages.append("target_link_libraries({} {}::{})".format(self.name, d.ref.name, d.ref.name))
   save(self, "conandeps.cmake", "\n".join(deps_packages))

Hope this helps someone else who might be new to all this too :)

[db4]

Hi @sully7

Yes, it is necessary to add find_package() calls in the CMakeLists.txt. This is exactly what that majority of users were pushing to have, a "transparent" CMake integration via find_package(). Do you mean that you would like a conandeps.cmake file that adds all the find_package() calls, so you can just included that file? In the worst case, this is something easily achievable with something like:

def generate(self):
     finds = ["find_package({} CONFIG REQUIRED)".format(d.ref.name) for d in self.dependencies.direct_host.values()]
     save(self, "conandeps.cmake", "\n".join(finds)

@memsharded This should take into account get_property("cmake_file_name") otherwise find_package() will fail. Can I access it in generate() method?

[db4]

Answering to myself: something like

        finds = []
        for d in self.dependencies.direct_host.values():
            name = d.cpp_info.get_property("cmake_file_name")
            if not name:
                name = d.ref.name
            finds.append(f"find_package({name} CONFIG REQUIRED)")
        save(self, "conandeps.cmake", "\n".join(finds))

seems to be enough

[memsharded]

Closing this as responded