incompatible_do_not_split_linking_cmdline: Order of C++ link flags affected

[oquenchil]

Flag: --incompatible_do_not_split_linking_cmdline
Available since: 0.25 (March 2019 release)
Will be flipped in: 0.26 (April 2019 release)
Tracking issue: #7670

This changes how flags are passed to the linker. If you don't have your own custom CROSSTOOL, then you don't need to migrate anything, but you should still test with this flag to see that nothing breaks. If it does, please let us know.

If you have your own custom CROSSTOOL:
If the CROSSTOOL doesn't support linker parameter files, then you don't have to migrate anything. If it supports linker parameter file but you do not specify the feature 'linker_param_file' in your CROSSTOOL, then you don't have to migrate anything.

If you specify the feature 'linker_param_file' and the linker is using gcc-style linker flags, then you should modify the feature so that instead of having a parameter file specified like this:
flag: '-Wl,@%{linker_param_file}'
it is like this:
flag: '@%{linker_param_file}'

In other words, the parameter file should not be passed directly to the linker with -Wl,. In addition to this you should specify the feature 'do_not_split_linking_cmdline' in your CROSSTOOL, whether it's enabled or disabled it does not matter, we only check its presence. This will tell Bazel to use the new behavior with this toolchain. After the flip of incompatible_disable_legacy_cc_provider this feature will do nothing, it can safely be deleted.

This change will put every flag inside the parameter file whereas before only flags starting with -l were put there, i.e.: all libraries. This might cause the order of link flags to change with respect to what you originally had in your CROSSTOOL, if it does you will have to re-order the corresponding flags. If you have to make any changes to the CROSSTOOL, make sure that you add the feature 'do_not_split_linking_cmdline' too. Example of change:

Old:

feature {
  name: 'linker_param_file',
  flag_set {
         action: 'c++-link-executable'
         action: 'c++-link-dynamic-library'
         action: 'c++-link-nodeps-dynamic-library'
         flag_group {
             expand_if_all_available: 'linker_param_file'
             flag: '-Wl,@%{linker_param_file}'
         }
}

New:

feature {
  name: 'do_not_split_linking_cmdline'
}
feature {
  name: 'linker_param_file',
  flag_set {
         action: 'c++-link-executable'
         action: 'c++-link-dynamic-library'
         action: 'c++-link-nodeps-dynamic-library'
         flag_group {
             expand_if_all_available: 'linker_param_file'
             flag: '@%{linker_param_file}'
         }
}

[dkelmer]

@oquenchil, there don't seem to be associated PRs submitted. Does the flag exist already? If yes, by when do you want people to migrate? If the flag exists, it will be available in the currently being cut release, 0.25. The earliest you can require people to migrate by is the 0.26 release. Is that the plan?

[dkelmer]

@oquenchil Seems like the flag does exist and that it was flipped (http://google3/third_party/bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppOptions.java?l=836&rcl=241059021). I think that the flag flip needs to be rolled back because there wasn't a migration window.

[hlopko]

You're looking into the wrong flag :) This is the change that introduced this flag: 3010e57. I'm changing the labels accordingly.

[dkelmer]

Oh, the title of the issue is inconsistent with the beginning of the content of the issue which is why I got confused :)
Thanks for updating the labels!

[hlopko]

Updated, sorry for the confusion.

[adzenith]

Enabling incompatible_do_not_split_linking_cmdline appears to put all of our linker flags after all libraries have been listed. This means that flags like --no-as-needed don't work, because they need to come before the libs. I can see how I can reorder the flags among themselves in our cc_config, but I'm not sure how to move them before or after the libs. Is there an easy way to do that?

Old order (desired):   Order with incompatible_do_not_split_linking_cmdline:
-rpath                 -rpath
-L                     -L
cc_config flags        -l
-l                     cc_config flags

[oquenchil]

Hey Nikolaus, there is no way to control the order except in cc_config.

[adzenith]

@oquenchil Is there a way in cc_config to do what I tried to show in my previous question? That is, can I put any of my flags before the list of libraries? If not, this is a breaking change for us that I don't know how we'll work around. We have this flag disabled for now. Should I open a new ticket?

[oquenchil]

Yes, you can control in cc_config the order of flags. But for custom flags that are coming in from individual rules with the attribute linkopts, they will all either before the list of libraries or they will all be behind. Unfortunately, there is no way to say these linkopts before libraries and these other linkopts after.

By cc_config I have assumed the whole time that you have your custom file like this:

bazel/tools/cpp/unix_cc_toolchain_config.bzl

Line 1026 in 8daa514

features = [

There you can change the order. For example there you can put user_link_flags_feature (linkopts attributes of rules) before libraries_to_link_feature.

[adzenith]

Yeah, I have a file similar to that, but I don't have any section that looks like this one

                                flags = ["%{libraries_to_link.object_files}"],
                                iterate_over = "libraries_to_link.object_files",

That is to say, I don't appear to have a feature anywhere that's actually adding the library paths to the linker command line. I assumed that something inside of Bazel was adding them automatically. I'm not sure how they're being added.

[oquenchil]

Some features get patched in by Bazel by default when they are not present in the config file. See:

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java

Line 626 in bd4ca0b

if (!existingFeatureNames.contains("libraries_to_link")) {

So all you have to do is copy over everything you see in CppActionConfigs.java between the ifLinux() brackets into your own config file. Or just copy it from the unix_cc_toolchain_config.bzl. Then once you have the libraries_to_link feature in the config file explicitly, just place it before or after any other feature you want.

[adzenith]

Awesome! Thanks so much for the help. I'll add it in.