1.10.0_1 build broke ABI w.r.t. to 1.10.0_0 builds #53

traversaro · 2022-10-25T17:42:24Z

See discussion in #50 .

@bluescarni while we discuss how to properly do the de-vendoring, I think that we can mark these packages as broken?

traversaro · 2022-10-25T17:53:23Z

@bluescarni while we discuss how to properly do the de-vendoring, I think that we can mark these packages as broken?

In case @conda-forge/spdlog agrees, the corresponding PR is ready to be merged at conda-forge/admin-requests#501 . Personally, I think the most important thing at the moment is fix the break that was induced by this change, then when everything is working we can think of how to properly de-vendor fmt.

traversaro · 2022-10-25T18:01:18Z

Hi @cav71, I may be wrong but it seems that this is more a comment on the de-vendoring in general? In that case, perhaps it could make sense to have a dedicated issue for the de-vendoring of fmt?
I am asking as this issue is dedicated just to the ABI breakage of 1.10.0_1 , and I think we can close it once we solve it, i.e. for example by merging conda-forge/admin-requests#501 .

cav71 · 2022-10-25T18:23:37Z

My bad, sorry: I pasted it the text in the wrong thread.

bluescarni · 2022-10-25T19:04:47Z

@traversaro I guess this is the only viable solution at this time.

It really sucks because the status quo (before this breakage) is such that it is impossible to use the current conda pinned fmt version (9.x) together with spdlog in the same package (or within a dependency chain). I guess I'll have to go and migrate back a bunch of packages to fmt pinned to 8.x :(

traversaro · 2022-10-25T19:25:57Z

Perhaps while we wait for an spdlog release we could think of doing a variant of this package with non-vendored fmt, but with a lower priority w.r.t. to the abi compatible vendored fmt spdlog . I would need to think about it, but perhaps we can find a solution before the new spdlog release.

bluescarni · 2022-10-25T20:20:09Z

@traversaro if such a strategy worked it would be a life saver... anything I can do to help? It sounds a bit beyond my conda-fu...

traversaro · 2022-10-27T18:00:12Z

@traversaro if such a strategy worked it would be a life saver... anything I can do to help? It sounds a bit beyond my conda-fu...

@bluescarni can you point me to one of the packages in which you want to install spdlog 1.10 and fmt 9.0 at the same time? So that I can try to get a clear idea on this.

kkraus14 · 2022-10-27T18:41:40Z

+1 to a variant with a track feature to weigh it down now and then consider moving to external fmt for 1.11 where we can more safely have an ABI break

traversaro · 2022-10-27T18:45:32Z

+1 to a variant with a track feature to weigh it down now and then consider moving to external fmt for 1.11 where we can more safely have an ABI break

The only missing piece of the puzzle for me is: how does the downstream package explictly depend on the custom variant? It may be obvious but I am not enough experienced with that.

bluescarni · 2022-10-27T18:48:15Z

Hi @traversaro thanks a lot for helping out...

So I have this package, heyoka, which depends on both spdlog and fmt:

https://github.com/conda-forge/heyoka-feedstock/blob/main/recipe/meta.yaml

As you can see from the recipe, I pinned spdlog to 1.10 and fmt to 8.1 in order to prevent ABI incompatibilities. So far so good.

The issue for the next version of heyoka if that one of its other dependencies, called mp++, in its latest version depends on fmt:

https://github.com/conda-forge/mppp-feedstock/blob/main/recipe/meta.yaml

The fmt version in this recipe is not pinned, so it currently depends on fmt 9.

This means that, as far as I have understood, it won't be possible to release the next version of heyoka until fmt is unbundled from spdlog, so that heyoka, spdlog and mp++ can all depend on the same version of fmt (i.e., the one globally pinned by conda-forge).

kkraus14 · 2022-10-27T19:55:02Z

The only missing piece of the puzzle for me is: how does the downstream package explictly depend on the custom variant? It may be obvious but I am not enough experienced with that.

I've seen two different approaches to it:

You have a selector package, i.e. something like the arrow-cpp-proc package that has cpu and cuda variants defined via build strings. The arrow-cpp package then has a run_constrained entry on that package, i.e. https://github.com/conda-forge/arrow-cpp-feedstock/blob/main/recipe/meta.yaml#L117
You just use build strings, i.e. here we could have bundledfmt and extfmt in the build string and a downstream package can then have a specification like spdlog=1.10=*extfmt*

h-vetinari · 2022-10-28T20:53:32Z

Bear is an example that doesn't depend on fmt, but breaks.

bluescarni · 2022-10-29T19:38:55Z

The only missing piece of the puzzle for me is: how does the downstream package explictly depend on the custom variant? It may be obvious but I am not enough experienced with that.

I've seen two different approaches to it:
1. You have a selector package, i.e. something like the `arrow-cpp-proc` package that has `cpu` and `cuda` variants defined via build strings. The `arrow-cpp` package then has a `run_constrained` entry on that package, i.e. https://github.com/conda-forge/arrow-cpp-feedstock/blob/main/recipe/meta.yaml#L117

2. You just use build strings, i.e. here we could have `bundledfmt` and `extfmt` in the build string and a downstream package can then have a specification like `spdlog=1.10=*extfmt*`

Would it make sense to create a temporary package called sdplog-unbundled (or something like that), to be removed/marked as broken after the new (hopefully unbundled) release of spdlog?

h-vetinari · 2022-11-02T00:23:22Z

New release is not far out, which means we could then do a clean break and re-migrate (it would be more work if it's only a patch version increment, but still feasible with some repodata patches).

h-vetinari · 2022-11-02T23:00:16Z

Heads up @conda-forge/spdlog

Upstream just tagged 1.11, and this repo is currently set to automerge. Since #50 was not reverted on main, that should be fine (we'll enter the brave new world as of 1.11), but JFYI

traversaro · 2022-11-02T23:34:20Z

Heads up @conda-forge/spdlog

Upstream just tagged 1.11, and this repo is currently set to automerge. Since #50 was not reverted on main, that should be fine (we'll enter the brave new world as of 1.11), but JFYI

Perhaps we want to disable automerge until #51 is fixed? @bluescarni

h-vetinari · 2022-11-03T00:22:34Z

I agree that the Debian patch would be good to have. The good thing is that we don't paint ourselves into a corner resolver-wise even if the automerge runs through. We can just add those patches afterwards, and it'll be pulled into the newer spdlog-as-runtime-dep.

bluescarni · 2022-11-03T08:14:50Z

Perhaps we want to disable automerge until #51 is fixed? @bluescarni

How does one disable automerge?

h-vetinari · 2022-11-03T08:43:21Z

How does one disable automerge?

Remove https://github.com/conda-forge/spdlog-feedstock/blob/main/conda-forge.yml#L1

bluescarni · 2022-11-03T08:57:13Z

How does one disable automerge?

Remove https://github.com/conda-forge/spdlog-feedstock/blob/main/conda-forge.yml#L1

Cheers, should be done now.

@kkraus14

This reverts commit 7bf7a76. cc: @kkraus14 There's a handful of issues this caused downstream. We'll be sorting through these and fixing them but need to quickly revert this for now. Currrent problems: 1. rmm's meta.yaml doesn't correctly specify spdlog as a build dep. It's [incorrectly listed as a runtime dep](https://github.com/rapidsai/rmm/blob/7bf7a763a3b6a32a7a57f900de35e2e5f30f17c0/conda/recipes/librmm/meta.yaml#L56). This means that rmm [pulls spdlog via CPM with rapids-cmake](https://github.com/rapidsai/rmm/actions/runs/3595940512/jobs/6056061203#step:6:595) `-- CPM: adding package spdlog@1.8.5 (v1.8.5)` and then clobbers its own installed spdlog headers: ``` 2022-12-01T19:46:46.2167513Z ClobberWarning: This transaction has incompatible packages due to a shared path. 2022-12-01T19:46:46.2168766Z packages: conda-forge/linux-64::spdlog-1.10.0-h924138e_0, file:///tmp/conda-bld-output/linux-64::librmm-23.02.00a-cuda11_gc328a18d_11 2022-12-01T19:46:46.2169686Z path: 'include/spdlog/async.h' ``` 2. spdlog 1.10.0 doesn't work with nvcc. We need spdlog 1.11.0 to get fmt 9.1.0, which contains [a necessary fix](fmtlib/fmt#2971). However, there's some uncertainty about how to handle the spdlog conda-forge package which [no longer vendors fmt](conda-forge/spdlog-feedstock#50) while rapids-cmake is still using the upstream repo which continues to vendor fmt. Does this inconsistency matter, as long as we pin `fmt=9.1.0` in the conda package cases? @kkraus14 It looks like you weighed in on this matter [here](conda-forge/spdlog-feedstock#53) and [here](conda-forge/conda-forge-pinning-feedstock#3654). If you have insights that can help us with this quandary, please share! Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Mark Harris (https://github.com/harrism) - Ray Douglass (https://github.com/raydouglass) URL: #1176

kkraus14 · 2023-10-07T04:04:08Z

This is now resolved

cav71 · 2023-10-07T09:03:53Z

Thanks!

This was referenced Oct 25, 2022

De-vendor fmt #50

Merged

Undefined symbol running rviz2 RoboStack/ros-humble#23

Closed

Mark spdlog builds 1.10.0-*_1 as broken to ABI breakage conda-forge/admin-requests#501

Merged

h-vetinari mentioned this issue Oct 28, 2022

current spdlog v1.x branch (b6051273) fails to build with fmt (b79ed41) or later gabime/spdlog#2485

Closed

kkraus14 mentioned this issue Nov 4, 2022

Update pin for spdlog conda-forge/conda-forge-pinning-feedstock#3654

Merged

2 tasks

bdice mentioned this issue Dec 5, 2022

Revert "Upgrade to spdlog 1.10 (#1173)" rapidsai/rmm#1176

Merged

3 tasks

kkraus14 closed this as completed Oct 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.10.0_1 build broke ABI w.r.t. to 1.10.0_0 builds #53

1.10.0_1 build broke ABI w.r.t. to 1.10.0_0 builds #53

traversaro commented Oct 25, 2022

traversaro commented Oct 25, 2022 •

edited

Loading

traversaro commented Oct 25, 2022

cav71 commented Oct 25, 2022

bluescarni commented Oct 25, 2022 •

edited

Loading

traversaro commented Oct 25, 2022

bluescarni commented Oct 25, 2022

traversaro commented Oct 27, 2022

kkraus14 commented Oct 27, 2022

traversaro commented Oct 27, 2022

bluescarni commented Oct 27, 2022

kkraus14 commented Oct 27, 2022

h-vetinari commented Oct 28, 2022

bluescarni commented Oct 29, 2022

h-vetinari commented Nov 2, 2022

h-vetinari commented Nov 2, 2022

traversaro commented Nov 2, 2022

h-vetinari commented Nov 3, 2022

bluescarni commented Nov 3, 2022

h-vetinari commented Nov 3, 2022

bluescarni commented Nov 3, 2022

kkraus14 commented Oct 7, 2023

cav71 commented Oct 7, 2023

1.10.0_1 build broke ABI w.r.t. to 1.10.0_0 builds #53

1.10.0_1 build broke ABI w.r.t. to 1.10.0_0 builds #53

Comments

traversaro commented Oct 25, 2022

traversaro commented Oct 25, 2022 • edited Loading

traversaro commented Oct 25, 2022

cav71 commented Oct 25, 2022

bluescarni commented Oct 25, 2022 • edited Loading

traversaro commented Oct 25, 2022

bluescarni commented Oct 25, 2022

traversaro commented Oct 27, 2022

kkraus14 commented Oct 27, 2022

traversaro commented Oct 27, 2022

bluescarni commented Oct 27, 2022

kkraus14 commented Oct 27, 2022

h-vetinari commented Oct 28, 2022

bluescarni commented Oct 29, 2022

h-vetinari commented Nov 2, 2022

h-vetinari commented Nov 2, 2022

traversaro commented Nov 2, 2022

h-vetinari commented Nov 3, 2022

bluescarni commented Nov 3, 2022

h-vetinari commented Nov 3, 2022

bluescarni commented Nov 3, 2022

kkraus14 commented Oct 7, 2023

cav71 commented Oct 7, 2023

traversaro commented Oct 25, 2022 •

edited

Loading

bluescarni commented Oct 25, 2022 •

edited

Loading