[question] Workspace with editable package in the middle of the dependency graph

[AndreyNautilus]

What is your question?

Hi! I have a relatively big project with dozens of packages maintained by multiple teams. Some teams want to have a conan workspace (we're still using conan-1) with editable package that they maintain, but without other packages. And I can't make it work.

For simplicity let's assume we have an app (a top-level package), which depends on static libraries liba and libb. libb also depends on liba. libb has tests (which transitively depend on liba) which run in build() method of its conanfile. A complete setup can be found here: https://github.com/AndreyNautilus/conan-workspace-playground

If I create a workspace with app and liba as editables:

editables:
    app/0.1@andrey/ws:
        path: app
    liba/0.1@andrey/ws:
        path: liba
layout: layout.ini
workspace_generator: cmake
root:
    - app/0.1@andrey/ws
    # note: liba is not a root!

then I can run conan workspace install and build the app and everything works. But in this setup liba is obviously not editable (changes to liba are not picked up when app is rebuilt).

If I add liba to root: of the workspace:

root:
    - app/0.1@andrey/ws
    - liba/0.1@andrey/ws

then conan workspace install starts to fail with:

[4/4] Linking CXX executable bin/b-unittests
FAILED: bin/b-unittests
ld: library 'a' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
libb/1.0@andrey/stable: ERROR: Package '788129619df7b2c84e11b27a0c60646a39512e05' build failed

That's a reasonable failure: liba is an editable package, it's not built yet, but libb already requires it. It looks like a chicken-and-egg problem. Is it even possible to achieve such setup with conan workspaces?

Dependency on liba in tests of libb is just one example. Another kind of dependencies could be that liba generates headers that libb uses.

I would like to have a command the can build the editables and inject them into the dependency group automatically. Is it possible?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @AndreyNautilus

Thanks for your question.

then I can run conan workspace install and build the app and everything works. But in this setup liba is obviously not editable (changes to liba are not picked up when app is rebuilt).

I am not sure I understand this. Are you sure you meant there liba or was it libb. Because in the setup liba is editable, isn't it?

I see that you are trying to use layout.ini and the cmake generators.
Those are very legacy, even inside Conan 1.X, not even considering they have been fully removed in Conan 2.
So I'd like to recommend to use the layout() method and the new CMakeDeps and CMakeToolchain generators. This is necessary for moving to Conan 2 anyway, so better prioritize moving to those. Please recall that Conan 2 is already like 20 months old, and ConanCenter packages will soon stop receiving updates for Conan 1, see https://blog.conan.io/2024/09/30/Conan-Center-will-stop-receiving-updates-for-Conan-1.html

If the workspace feature in Conan 1.X is not working fine with the layout() and new CMakeDeps generator, I'd recommend to skip it at the moment. The workspace feature was dreopped in Conan 2 because of being a bit fragile and not enough flexible feature. While we are working to bring the workspace feature back to Conan 2, track this issue #15992 to track progress there.

In the meantime, it might be more pragmatic and effective to use a script that manages editable packages.

[AndreyNautilus]

Hi @memsharded , thanks for your reply.

then I can run conan workspace install and build the app and everything works. But in this setup liba is obviously not editable (changes to liba are not picked up when app is rebuilt).

I am not sure I understand this. Are you sure you meant there liba or was it libb. Because in the setup liba is editable, isn't it?

I meant "even though liba is listed in editables: section, it's not considered editable when conan workspace install is called". When I run conan workspace install and inspect the logs I see:

Packages
    liba/1.0@andrey/stable:dadb10329df2d46b840375ac035bcc7550700723 - Cache
    libb/1.0@andrey/stable:91713887aee2a38782addfb94e10bc293bff6cb3 - Cache
    app/0.1@andrey/ws:26ccd763d84debe4a7683c775a95416b44f74902 - Editable

see: liba is taken from cache, it's not editable. ~/.conan/data/liba/1.0/andrey/stable/package/dadb10329df2d46b840375ac035bcc7550700723/ is a regular folder, not a symlink. And code changes in liba folder are not picked up when the workspace is built.

I see that you are trying to use layout.ini and the cmake generators. Those are very legacy, even inside Conan 1.X, not even considering they have been fully removed in Conan 2. So I'd like to recommend to use the layout() method and the new CMakeDeps and CMakeToolchain generators.

Yeah, that's quite an old setup for an old project. We have migration to conan-2 on our radar, but I can't influence the schedule, so I'm stuck with this old setup for now.

We can switch to layout() method and new generators, but it seem orthogonal to the topic.

In the meantime, it might be more pragmatic and effective to use a script that manages editable packages.

I wanted to avoid manual scripting, but it seems to be the only possible way forward.

[memsharded]

But it seems there is some discrepancy there. The workspace contains liba/0.1@andrey/ws, but your app depends on liba/0.1@andrey/stable, not on the one on ws channel, this is why it might be taking it from the cache?