Propagate CXX version used to compile abseil

[h-vetinari]

Fixes #1116

Second commit optional

[Lindurion]

@h-vetinari not sure how much of the past context you've read through on this topic, but I don't think this change would enforce the C++ standard ABI constraint that Abseil really wants to express. It's been awhile, so please correct me if I'm missing something.

When I last get into the weeds on this, my conclusion was that it would really require a new CMake feature to enforce a same C++ standard constraint (as opposed to minimum C++ standard). I filed https://gitlab.kitware.com/cmake/cmake/-/issues/22592 as a feature request, which has received no attention (as evidently not that common of an issue for typical C++ projects).

I think with your proposed change, for example, nothing would prevent a downstream target from setting a higher cxx_std_NN value than what CXX_STANDARD (and thus ABSL_CXX_STANDARD) was at the time Abseil gets built, and then that would hit the same ABI compatibility issues.

I think cxx_std_NN is best used as CMake currently intends it--to express the minimum C++ version a target requires, and the best we can do around enforcing same C++ standard ABI compatibility requirement right now is documentation. Does that make sense?

[h-vetinari]

Does that make sense?

Yes it does, though the way I understand things, the minimum might actually be enough.

The ABI issues come from abseil types not being ABI-compatible with the later stdlib-types they are intended to backport. However, even a newer standard can use the backported types (rather than the stdlib ones), which is why absl/base/options.h allows setting the usage of the compat types even for higher standards.

[h-vetinari]

CC @coryan since I've seen you propose this in a few places in recent discussions. :)

[coryan]

Is ABSL_CXX_STANDARD always defined? I think the default is set here:

abseil-cpp/absl/copts/AbseilConfigureCopts.cmake

Line 97 in 4a1ccf1

set(ABSL_CXX_STANDARD "${CMAKE_CXX_STANDARD}")

And that looks like it can be the empty string? And if is the empty string, then the C++ standard in effect depends on the compiler (and compiler version).

[Lindurion]

@coryan see the example patterns in the Abseil CMake README here:
https://github.com/abseil/abseil-cpp/tree/master/CMake#step-by-step-instructions

I think library authors (including Abseil itself) should NOT set CMAKE_CXX_STANDARD, which should be left up to the end application project.

When a library project is being directly developed or CI tested, then it can safely set those things without affecting its clients (which is the pattern the if(CMAKE_SOURCE_DIR STREQUAL my_lib_project_SOURCE_DIR) check in the 2nd README example shows; it can also be set via the cmake command-line in build scripts).

[Lindurion]

Yes it does, though the way I understand things, the minimum might actually be enough.

I don't think so, but maybe a concrete example would help work through the details here--do you have a specific scenario that is currently broken but your change would cause to build correctly? Are you seeing issues in an application project or in a library that other clients then depend on?

Guess I have 2 concerns here:

1. Philosophical: Modern CMake has intended semantics around cxx_std_NN and CMAKE_CXX_STANDARD, which this would no longer be following.
2. Practical: I don't think this change would actually help?

On philosophical semantics, I still think the bullet point list in this issue comment is the right approach re: what Abseil should set as a library author and what end application projects are in control of through CMAKE_CXX_STANDARD.

Practically, when a given target is built, the C++ standard used is the maximum of cxx_std_NN (for the target and its dependencies) and CXX_STANDARD (default initialized to CMAKE_CXX_STANDARD, which ABSL_CXX_STANDARD is just an alias for). It seems like this proposed change would raise cxx_std_NN to match a higher CMAKE_CXX_STANDARD, but that higher CMAKE_CXX_STANDARD should already be winning to determine the standard the Abseil library targets are built with.

This also wouldn't help the scenario described in my CMake feature request, since another depended upon library (another::lib in that example) can always have a higher cxx_std_NN value than the current value of CMAKE_CXX_STANDARD and cause different targets to be built with different C++ standards.

[h-vetinari]

It seems like this proposed change would raise cxx_std_NN to match a higher CMAKE_CXX_STANDARD, but that higher CMAKE_CXX_STANDARD should already be winning to determine the standard the Abseil library targets are built with.

The way I see the situation: an abseil built with (say) C++11 can be used in a downstream project compiling with CMAKE_CXX_STANDARD=17 (even though it's not recommended, and the maintainers here have stated that the only really supported case is a consistent C++ version across the entire stack).

But if abseil is built with C++17, a downstream project consuming that artefact may never go below that version (because it will not have the stdlib symbols that abseil compiled in this way will use by default; edits to absl/base/options.h notwithstanding).

The point is that the abseil artefacts generated by a given build configuration should encode this constraint, because the alternative (to a clearly actionable "compile with a higher C++ standard" message from the build system) are obscure linker errors that make diagnosing the correct cause of the issue very difficult.

I've been there on many projects within conda-forge, where I help package things.

[coryan]

In general, I think Abseil should be changed such that the installed artifacts reflect their requirements.

What does that mean?

• If Abseil was compiled with C++14 (or lower with older versions of Abseil) then:
  • absl/base/options.h should set ABSL_OPTION_USE_STD_* to 0 because that is the only value that will work.
  • The target_compile_options() should say cxx_std_14 (or cxx_std_11 with older version), because anything C++ version >= 14 will work.
• If Abseil was compiled with C++ >= 17 then:
  • absl/base/options.h should set ABSL_OPTION_USE_STD_* to 1 because that is the only value that will work.
  • The target_compile_options() should say cxx_std_17, because only C++ versions >= 17 will work.

I think this change is not quite right. It does not change the absl/base/options.h header. It assumes that CMAKE_CXX_STANDARD is set, which may not be (aside: the only way I know to reliably detect the C++ version in effect is to compile some C++ code that checks what version is currently in effect).

@h-vetinari sorry if I mislead you in the Conda discussions. The situation there is different because the package recipes have more control over the environment. Abseil operating in the wild needs to be

[Lindurion]

@coryan @h-vetinari I think there are distinct issues here:

[A] Before being built, what requirements can/should Abseil targets express through the CMake build system?
[B] After being built, what requirements can/should built Abseil targets express through packaging and linking?

I'm arguing that target_compile_options() set to cxx_std_14 is strictly in bucket [A], since it says: this target can only be built with a C++ standard of 14 or later, i.e. which versions of the C++ standard it is allowed to be built with in the first place. I don't think that mechanism is designed to directly express anything about [B].

The absl/base/options.h headers do clearly document that only values of 0 and 1 should be used if you are distributing Abseil in a binary package manager. But the default CMake build configuration should assume that Abseil is being built from source (e.g. as a CMake subproject included through FetchContent).

It sounds like you want Abseil's CMake configuration to make properly packaging it in binary distributions through cmake install easier? Right now it doesn't look like there's any logic around automatically setting ABSL_OPTION_USE_STD_* etc.

But as far as I know, setting cxx_std_NN doesn't propagate anything that helps you in world [B]. (And it's not the final word on the C++ standard the target is actually built with, which is what you really need to know anyway).

[coryan]

[A] Before being built, what requirements can/should Abseil targets express through the CMake build system? [B] After being built, what requirements can/should built Abseil targets express through packaging and linking?

I'm arguing that target_compile_options() set to cxx_std_14 is strictly in bucket [A], since it says: this target can only be built with a C++ standard of 14 or later, i.e. which versions of the C++ standard it is allowed to be built with in the first place. I don't think that mechanism is designed to directly express anything about [B].

Maybe. But it is clearly insufficient too.

In the same project, even if using FetchContent(), you could have a separate library or program that uses Abseil and sets target_compile_features(target-that-uses-abseil PRIVATE cxx_std_17). That would be a break if the compiler defaults to C++14 and therefore Abseil was compiled with C++14.

Conversely, (and this is more contrived) with some compilers the default is C++17. So, Abseil may have been compiled with C++17, but it currently sets its target_compile_features() to cxx_std_14. If a downstream targets has the CXX_STANDARD property to 14 then that target would be compiled with C++14. That would break too.

The absl/base/options.h headers do clearly document that only values of 0 and 1 should be used if you are distributing Abseil in a binary package manager. But the default CMake build configuration should assume that Abseil is being built from source (e.g. as a CMake subproject included through FetchContent).

I am not sure what the default CMake build configure should assume or not. What I am sure of, is that the artifacts installed by Abseil are very fragile. They break when the consuming project uses a mismatched C++ version. I know that Abseil's documentation says that one should not do this. But Abseil's behavior is surprising, it leads to breakage (see #696), and could be improved upon.

It sounds like you want Abseil's CMake configuration to make properly packaging it in binary distributions through cmake install easier?

I would like Abseil to install artifacts that are safe to use. Currently it installs artifacts that cause trouble. You could argue that should be the work of the package recipes. Maybe, but perusing #696 shows that the current situation causes more trouble than not.

I think I already articulated how that it is possible (maybe not easy) to provide safer installed targets:

• Patch the options.h file when installed to reflect the actual ABI
• Set the target_compile_features() to reflect the actual C++ standard used to compile Abseil.

If what I described somehow causes trouble for you [A] scenario please show how. I don't think it does, and in fact I think it improves that scenario too.

But as far as I know, setting cxx_std_NN doesn't propagate anything that helps you in world [B]. (And it's not the final word on the C++ standard the target is actually built with, which is what you really need to know anyway).

AFAIK it does. The installed CMake configuration files include the target_compile_features() (as properties). A downstream target using the installed artifacts via find_package() will load those options too.

In any case, this discussion is moot. This PR is not sufficient as-is.

[h-vetinari]

Thanks for the insightful comments!

In any case, this discussion is moot. This PR is not sufficient as-is.

The PR hasn't been touched in half a year and makes no claims to completeness. It's also easy to fix once there is a direction of what to do, and the discussion for that direction is not moot IMO.

[coryan]

Thanks for the insightful comments!

In any case, this discussion is moot. This PR is not sufficient as-is.

The PR hasn't been touched in half a year and makes no claims to completeness. It's also easy to fix once there is a direction of what to do, and the discussion for that direction is not moot IMO.

Fair enough. Let me try to implement my proposals above.

[Lindurion]

Maybe. But it is clearly insufficient too.

Agreed on that. Just a few more thoughts here:

To restate big picture, it sounds like you (and many other clients) want the Abseil project to be more opinionated + helpful about how it gets packaged through its CMake build + install support.

In that scenario, Abseil is being built directly, so it will be the top-level CMake project, right? It should probably be more opinionated about CMAKE_CXX_STANDARD and use this pattern itself to establish a stable functioning default, if none was configured by the building user/script:

# Leave C++ standard up to the root application, so set it only if this is the
# current top-level CMake project.
if(CMAKE_SOURCE_DIR STREQUAL absl_SOURCE_DIR)
  set(CMAKE_CXX_STANDARD 14)  # Or 17? or 20?
  set(CMAKE_CXX_STANDARD_REQUIRED ON)
endif()

Alternatively it could fail to build if none was set and link to documentation around it.

Setting cxx_std_NN per target to match CMAKE_CXX_STANDARD still feels weirdly circular to me, but I understand your motivation. (It should at least also check that CMAKE_CXX_STANDARD >= 14 as the minimum supported version, if this does end up being the best agreed upon solution).

All these other details (e.g. cxx_std_NN) would of course be more straightforward if Abseil had separate targets for different C++ standards. This happens anyway as soon as Abseil gets built + packaged with prebuilt binaries (I saw some questions around whether those prebuilt artefacts need to at least encode C++ standard into artefact names). @derekmauro: curious if there is openness on the Abseil team here to getting more involved in how Abseil is installed and packaged? Is the idea of separate build targets per C++ standard out of the question?

It seem like Abseil itself could take a few positions on this, such as:

• Current state: only really support bazel/google3-style build the world from source with one compiler toolchain configuration. The CMake build tries to help (in e.g. FetchContent() subproject scenarios), but CMake's model is sufficiently different and doesn't really have built-in mechanisms to enforce Abseil's documented ABI requirements, so hitting unhelpful linker errors is common. Then packaging scripts are basically on their own to fix Abseil up (including cumbersome steps like patching options.h) and distribute it in unsupported configurations.
• Minimal packaging-friendly improvements: at least set a default CMAKE_CXX_STANDARD when Abseil is the top-level CMake project (as I mentioned above) and improve options.h to be code generated in the Bazel and CMake builds/installs so that no post-install source file patching is required to package a functioning prebuilt Abseil library.
• Export a default set of installed packages: provide valid combinations of C++ standard, options.h values, etc. and include in Abseil's own CMake code this default set of sane configurations that packagers (or one-off install scripts as in Abseil should install w/ the correct base/options.h #696) can more safely use.
• Split targets up higher in the builds: even for the build-the-world-from-source case, in both the Bazel and CMake builds, split targets up by e.g. absl-cpp14, absl-cpp17, absl-cpp20, and maybe the current build-from-source-and-detect model is could be kept as the default, but dropped from the install set (since it doesn't work well when prebuilt)? This doesn't seem that crazy to me for a library with such a C++ standard backfill role.

AFAIK it does. The installed CMake configuration files include the target_compile_features() (as properties). A downstream target using the installed artifacts via find_package() will load those options too.

Got it, I was missing this aspect before.

Is there an easy way to override these properties more explicitly per installed target (when hopefully you know more about the actually built versions)? I guess the best way to do this also largely depends on the higher-level question above.

[coryan]

FWIW, I created a branch with the changes that (I believe) fix #696:

master...coryan:abseil-cpp:fix-installed-artifacts-reflect-their-ABI-requirements

I am happy if those changes get copied here, or just serve as inspiration. I am also happy to start my own PR. In the last case, as a Googler, I will be asked to make the change in the internal Google SCM and that will get exported "eventually".

[derekmauro]

FWIW, I created a branch with the changes that (I believe) fix #696:

master...coryan:abseil-cpp:fix-installed-artifacts-reflect-their-ABI-requirements

I am happy if those changes get copied here, or just serve as inspiration. I am also happy to start my own PR. In the last case, as a Googler, I will be asked to make the change in the internal Google SCM and that will get exported "eventually".

If you sent those changes to me as a Piper CL I would accept them (maybe after I tested them).

By the way, exports from Piper to GitHub now happen automatically in minutes.

[coryan]

If you sent those changes to me as a Piper CL I would accept them (maybe after I tested them).

Sounds good. I think I will have a CL by the end of today.

By the way, exports from Piper to GitHub now happen automatically in minutes.

TIL.

[coryan]

@h-vetinari I think you can safely close this PR. My changes got pushed as part of 308bbf3