[question] Is it possible to select different generators for different packages?

[artem-ogre]

We would like to use Conan with a CMake project.

For dependencies that provide package-config file or a find-module file (<Package>Config.cmake, Find<Package>.cmake) we would like to use cmake_paths generator.

In other cases we would like to generate find-module files with cmake_find_package generator.

In a mixed situation the solution is to use both cmake_paths and cmake_find_package generators and explicitly tell CMake which mode to use for find_package:

find_package(pcf_package CONFIG REQUIRED)
find_package(fm_package MODULE REQUIRED)

Another option is to use CMAKE_FIND_PACKAGE_PREFER_CONFIG.

But wouldn't it be great instead to tell Conan which generator to use for which package?
Something like:

[generators]
cmake_find_package # default
packageA:cmake_paths # prefer package A's package-config file

Question:
Is something like this possible?
Is there a better way to handle such mixed situations?
If not, would that be something worthy of a feature-request?

Thank you!

[jgsogo]

Hi!

Right now this is not something possible. You can always write a conanfile.py, declare both generators and remove the files you don't need from the install folder, but it is more a hack than a workaround.

This would be a new feature, to be implemented the same way you proposed, as settings and options currently allow that syntax in profiles.

[memsharded]

I don't think it is just a syntax issue, I see a problem with this, and is transitivity.

For example, the generator that we are most likely using as default in Conan 2.0 is cmake_find_package_multi (probably with a new name?). This generator is transitive, if you find one package, it will bring the information from the transitive dependencies as well, to not force the consumers to list the complete subgraph of things that they didn't declare to use.

That generator relies on the other packages to have cmake_find_package_multi generated files as well. It doesn't seem possible to opt-out for them, and analyzing the logic of which generators are necessary for each package, seems a bit too complicated.

[artem-ogre]

Thanks for the prompt replies, Javier and James.

Yes, I can see how new syntax could cause problems as it was not foreseen from the beginning.
Maybe I shouldn't have start with the suggestion but rather with the problem. You could recommend a better solution with the deep knowledge you have.

So maybe I should start over :)

Problem

When cmake_paths and cmake_find_package generators are used together: autogenerated FindXXX.cmake shadows XXXConfig.cmake (when a dependency provides one).

Goal

To use XXXConfig.cmake instead without changing anything in CMakeLists.

Why

Imagine existing project that uses a combination of FindXXX.cmake and YYYConfig.cmake for different dependencies. It would be nice to migrate such a project to Conan without the need to change anything in CMakeLists. Using autogenerated FindXXX.cmake instead of custom ones which were doing pretty much the same wrapping is an added bonus.

Please let me know if I could provide any more clarity to the problem.

[jgsogo]

IMO, it is up to the user to fix these errors once they are raised. Right now, the syntax in profiles (and CLI) allows the user to mix different settings (some combinations won't link or work at all). Same transitivity problems.

For coherence, Conan should take care of these problems for all fields or for none of them, should allow the same syntax for all the fields or for none of them (less surprise). Probably for none: if the user is mixing settings or mixing generators and gets an error related to building/linking or to generators, the user should realize about it and revert this opt-in behavior.

In some scenarios (like this one), using different generators for certain packages could be the way to go.

[tjwrona]

I agree with @artem-ogre, I am using CMake projects and simply wrapping them into conan packages that work very intuitively with the "cmake_paths" generator. (They install their own package config files correctly and can easily be found using the "cmake_paths" generator)... Also transitive dependencies seem to work fine this way too. Some of my CMake projects use conan to consume other packages and those dependencies get carried along fine.

But the second I need to include a third party library that doesn't support "cmake_paths", I need to add the "cmake_find_package" generator and it breaks the libraries that use "cmake_paths" because of the problem Artem mentioned above.

I tried making "cmake_find_package" compatible packages from these projects, but compiler options like setting the correct C++ standard do not get transitively resolved properly and I have to add duplicated information in the conanfile that is already in the CMakeLists.txt to get things to work (this becomes a maintenance problem). With the "cmake_paths" generator these things do get transitively resolved without any extra effort because the actual CMake target produced by CMake itself is consumed when using the package. (see here: #7479)

I really prefer the simplicity of just wrapping an existing CMake project and using its own package config files to find it with "cmake_paths". Even though it restricts me to only consuming my libraries with CMake, that doesn't bother me because I have no intention of switching build systems. It seems to be the only way things just work without any headaches.

[CD3]

I agree. I am doing the same thing. For my projects, I am writing recipes that just do a normal cmake install and then set the environment variable MyProject_DIR to the package directory. Using the virtualenv generator, everything works like it would if it had been installed "by hand".

For third-party packages, the the new CMakeDeps generator is nice, but it interferes with package-installed cmake config files. It would be nice to have a way to have CMakeDeps generate config files for only some of the packages (the ones that need it) and still use the package installed config files.

[CD3]

I've been experimenting with different ways using a packages CMake Config file and I have found that this works if you have control over the recipe.

def package_info(self):
    package_path = pathlib.path(self.package_folder)

    self.cpp_info.set_property("cmake_find_mode","none")  # tell CMakeDeps generator not to write a Config file
    self.cpp_info.set_property("cmake_target_name", "MyLib::MyLib")  # this needs to match what the libraries Config file will actually define
    self.cpp_info.builddirs = [ "cmake" ] # tell CMakeToolchain generator to add the packages cmake file to `CMAKE_PREFIX_PATH` and friends

I've also tried

def package_info(self):
    package_path = pathlib.path(self.package_folder)
    self.cpp_info.set_property("cmake_build_modules", [ "cmake/MyLibConfig.cmake" ]

which will add a line to one of the CMakeDeps generated files that just sources the package's Config file. This isn't very nice through because then both CMakeDeps and the package's Config are trying to setup the target. You can set the cmake_target_name to something different so that it doesn't get used, but what if there was a configuration setting for CMakeDeps that said "generate a Config file, but inside, just source these files"?

However, neither of these workarounds support multi-config, which is what I'm working on now. In the first case, the toolchain file will be overwritten each time conan install is ran, so it will point to whichever config was installed last. In the second case, both configs get sources when cmake is ran, so whichever one runs last wins. CMakeDeps does mult-config by wrapping all the settings passed to set_properties(...) in generator expressions, but that would require the project's Config file to provide the support.

[lasote]

"generate a Config file, but inside, just source these files"?

No, there is not that possibility. The config files in your package should be located with a call to find_package directly so the first approach looks better. Including a "config" from another config generated by conan looks weird.

About the multi-config, I'm aware of the limitation of the CMakeToolchain that currently sets the CMAKE_PREFIX_PATH/CMAKE_MODULE_PATH to the latest installed.

But what would be the expected behavior? I mean, cmake cannot do an include() with a generator expression depending on the build_type so, what would happen if we manage to "accumulate" all the .builddirs declared for the (Debug/Release...) and include them all? Is that what you are expecting? I guess not, right? that would make available several xxx-Config.cmake from your different configurations but eventually only one would be located, right?

Have you figured out any other way to manage the multi-config only with the packaged config files?

Thanks

[CD3]

Including a "config" from another config generated by conan looks weird.

I agree, and we actually have a package that depends on MyLib_DIR being set to the correct directory, which won't be the case if CMakeDeps generates one in the build directory.

But what would be the expected behavior? I mean, cmake cannot do an include() with a generator expression ...

You are correct. I am not very familiar with generator expressions, so I was still trying to understand how it could work. Generator expressions only affect cmake's "generate" step so they cannot conditionally include a file, but they can conditionally link against targets.

I was able to get something to work, but it is a bit of a hack, and it requires the second approach above. Let's say you have a library that include a xxx-Config.cmake that will define a target named MyLib::MyLib.

1. In the package(self) method: After installing the package with the conan.tools.cmake.CMake helper, edit all of the installed cmake files and rename the target from MyLib::MyLib to MyLib::MyLib_<CONFIG> (i.e. MyLib::MyLib_DEBUG).
2. In the package(self) method: Write small CMake file that will load the packages xxx-Config.cmake file, define the MyLib::MyLib targets as an interface library, and link against MyLib::MyLib_<CONFIG> with a generator expression.
3. In the package_info(self) method: Set the cmake_target_name to something that will not be used (like IGNORE::MyLib) so that the CMakeDeps targets won't be used.
4. In the package_info(self) method: Add the script written in step 2 to cmake_build_modules so that it will be sourced.

It looks like this

    def package(self):
        cmake = CMake(self)
        cmake.install()

        # Add support for CMakeDeps multi-config...
        # We want to use the CMake Config file that gets installed by the package
        # because it already knows all of the configuration settings.
        # However, we also want to support multi-configs.
        # So...

        # Get config type strings in various cases. i.e. Debug, debug, DEBUG
        # This is just for convenience.
        Config = str(self.settings.build_type)
        config = Config.lower()
        CONFIG = Config.upper()
        loader_content = []

        # Get the directory that our cmake files are installed in
        CMake_DIR = next(pathlib.Path(self.package_folder).glob(f'*/cmake/'))

        # Rename targets to include the config.
        # i.e. MyLib::MyLib -. MyLib::MyLib_DEBUG
        for file in CMake_DIR.glob(f'*.cmake'):
            content = load(file)
            content = content.replace("MyLib::MyLib", f"MyLib::MyLib_{CONFIG}")
            save(file,content)

        # Write a file that will create the original targets (MyLib::MyLib in the example above)
        # and link it to the <TARGET>_<CONFIG> target with a generator expression. Thay way,
        # the actual targets that get linked in will depend on the config.
        loader_content.append( f'''
        if( NOT TARGET MyLib::MyLib )
        add_library( MyLib::MyLib INTERFACE IMPORTED )
        endif()

        message(STATUS "Including ${{CMAKE_CURRENT_LIST_DIR}}/MyLibConfig.cmake")
        include("${{CMAKE_CURRENT_LIST_DIR}}/MyLibConfig.cmake")
        target_link_libraries( MyLib::MyLib INTERFACE $<$<AND:$<CONFIG:{Config}>,$<TARGET_EXISTS:MyLib::MyLib_{CONFIG}>>:MyLib::MyLib_{CONFIG}> )
        ''')


        save(CMake_DIR/"MyLib_LoadConfig.cmake","\n".join(loader_content))

    def package_info(self):
        package_path = pathlib.Path(self.package_folder)

        # We don't want to use *any* of the Conan generated targets, so we have it generate
        # a target name that is different than the "real" target name.
        # Maybe there is an option to not generate the targets at all?
        self.cpp_info.set_property("cmake_target_name", "IGNORE::MyLib")
        # Now list our loader script above in the build_modules so that it will get included when we run cmake.
        self.cpp_info.set_property("cmake_build_modules", [ str(p) for p in package_path.glob("*/cmake/MyLib_LoadConfig.cmake") ])

This almost works with Conan's CMakeDeps generator, except that Conan will only source cmake_build_modules from one package. The bottom of the generated MyLibConfig.cmake file looks like this:

# Only the first installed configuration is included to avoid the collision
foreach(_BUILD_MODULE ${MyLib_BUILD_MODULES_PATHS_RELEASE} )
    conan_message(STATUS "Conan: Including build module from '${_BUILD_MODULE}'")
    include(${_BUILD_MODULE})
endforeach()

So I wrote a custom generator that just uses CMakeDeps and edits this file to include both RELEASE and DEBUG.

This actually works for the Ninja Multi-Config and Visual Studio cmake generators on the project I am testing, but I am not sure how general it is. If a packaged xxx-Config.cmake relied on reading and setting global variables, I could see there being potential for conflicts arising. I am planning on trying it out on a few more projects and seeing what happens.

[memsharded]

@czoido is going to propose some client side (from CMakeDeps) configuration to customize how dependencies are consumed ("generated config.cmake", "generated Cmake Find module", "in-package cmake file").