[workspaces] Decoupled Workspaces

[wpalfi]

We started using conan for our own libraries more than two years ago. From the very beginning, we have often been working on multiple libraries (packages) in one visual studio solution, long before conan editables and workspaces were introduced, just using cmake's add_subdirectory and find_package. Now I tested conan workspaces for the first time. They are promising and similar to our solution, but there are still important issues that discourage me from using them. I want to share my feelings here:

Decoupling

My main concern is the coupling between conan and source. Strictly separating the library source from all the conan stuff makes things much easier to understand and more flexible. I am not only talking about the library structure, but also about how libraries are linked in a multi library workspace, see below.

CMake Only

I created an example https://github.com/wpalfi/conan-decoupled-workspace based on https://github.com/conan-io/examples/tree/master/features/workspace/cmake for illustration.

Let's start with the simplest cmake based solution, no conan yet. We are using only the source directories chat/src, hello/src, say/src. Each of the three source directories has a conan free CMakeLists.txt. It follows modern cmake's best practices and is reduced to the minimal calls to add_library, find_package and target_link_libraries. That's extremely important to be able to consume the library later with add_subdirectory in a flexible way.

project(Hello)
add_library(hello hello.cpp)
target_include_directories(hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

add_library(hello::hello ALIAS hello)

if(NOT TARGET say::say)
    find_package(say REQUIRED)
endif()
target_link_libraries(hello say::say)

Each of the three libraries has a root CMakeLists.txt with the required add_subdirectory calls to build the respective library together with its dependencies.

project(Hello)
add_subdirectory(../say/src say) # could use e.g. cache variable for path in real world
add_subdirectory(src)

So the layout of each library is

• hello
  CMakeLists.txt -> add subdirectories
  • src
    CMakeLists.txt -> add, find and link targets
    hello.cpp, hello.h

It is good practice in software development to always work on the smallest possible subset of code. Therefore, using the src folder should be preferred over taking the indirection via the outer folder.

Adding Conan On Top

Now we want to create conan packages of the three libraries without touching the source directories. In real world, the conan stuff could also live in separate repositories.

We add conanfiles and modify the root CMakeLists.txt of each library to

project(Hello)

option(HELLO_WITH_CONAN "Use conan" ON)
if(HELLO_WITH_CONAN)
    include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
    conan_basic_setup(TARGETS)
else()
    add_subdirectory(../say/src say) # could use e.g. cache variable for path in real world
endif()

add_subdirectory(src)

In order for this to work, we need to tell CMake how to find the packaged libraries. We dont want to touch the src folder, so we need a custom find module to switch between source target and conan target:

findhello.cmake:

if(TARGET hello::hello)
    set(hello_FOUND TRUE)
elseif(TARGET CONAN_PKG::hello)
    # create hello::hello and link to conan packet
    add_library(hello::hello INTERFACE IMPORTED)
    target_link_libraries(hello::hello INTERFACE CONAN_PKG::hello)
    set(hello_FOUND TRUE)
else()
    set(hello_FOUND FALSE)
    if(hello_FIND_REQUIRED)
        message(FATAL_ERROR "Unable to find hello")
    endif()
endif()

So we created a way to switch between using conan (top row in drawing below) and building from source (bottom row), for each library separately.

E.g. there are three different ways to build chat.

• build all
• say from conan
• say and hello from conan

For mixed conan/source builds, an adjusted conanfile is needed. E.g. to build chat together with hello, but with say from conan, we would need to remove the hello requirement and add a say requirement instead. (Usually it is easier to use a conanfile.txt here.)

Building chat together with say but with hello from conan is trickier. (There is no connection from hello-conan to say in the drawing.) Conan editables can be used but that's a poor solution with the say library actually existing twice with all possible problems:

Remember that all this was without workspaces and (almost) without editables. The main issues of this approach are

1. Need to add some lines for add_subdirectory to root CMakeLists.txt for each library that may be edited (can be a lot in real world!).
2. Need to adjust conanfile (fix requirements)
3. No (good) way to build say without building hello

Workspaces

Conan workspaces already solve 1 and 2 :-).

3 could also be solved if workspaces would link directly to the cmake targets created with add_subdirectory. Currently they are using editables with the problems mentioned above.

left: CMake tree created by conan workspace, right: this is what we want

Even with issue 3 solved, workspaces are currently not flexible enough, even for this simple example. They are designed to just put multiple conan builds together. Each of these built libraries must have its own conanfile, build folder and conan_basic_setup. For the common use case of having a single root application or library, it should be sufficient to have a single conan install and conan_basic_setup in the root folder. Libraries that are built from source, should be automatically added with add_subdirectory (using a conan free source directory) and directly linked without editables (should not even require a conanfile). There should be no need to create a separate workspace folder, as the add_subdirectorys should be called from the root application/library folder.

[x] I've read the CONTRIBUTING guide.

[memsharded]

#15992 is centralizing effort and feedback for the Conan 2 workspace feature, I am closing this one, lets follow up there.

Also, some of the stuff in this thread would be obsolete, like conan_basic_setup() that no longer exists. Lets wait for the kickoff of workspaces 2.0 in that ticket and then we can iterate from there.