Design review for Platform-based Flags

[katre]

Another design from Platform Features API, this is for platform--based flags, which aims to obsolete half of the platform mappings file, and unlock new capabilities, by adding a new flags attribute to the existing platform rule, listing flags and values to enable when the platform is the target platform.

See the full design and let me know what you think. Specifically, I'm curious about what is the least confusing behavior when the same flag is present in the platform and on the command line, and whether that changes if the platform is also set on the command line, or via a transition.

[fmeum]

I like the general direction of this, moving flags into BUILD file and linking them to labeled entities.

This brings up a few more questions about flag precedence: If a transition changes both --platforms and a flag also set by the new platform, which value wins? If a flag is accumulative (e.g. --copt), in which order are the values inserted relative to command-line arguments and values specified by transitions?

[katre]

These are great questions.

As a user, what would be least surprising to you?

Considering transitions, where the transition and the platform both specify the same flag, but with different values:

1. The flag value set directly in the transition is used.
2. The flag value from the platform is used.
3. The value used depends on whether --platforms is before or after --flag in the transition.
   1. I included this one for completeness but want to note that I hate it.

Considering command line flags, where the user specifies both --platforms and --flag:

1. In this case, is flag order important?
2. If the exact flag always overrides, is that confusing?
3. What about bazelrcs?

At this point I'm listing questions, I'd love to know what actual users think would be useful and least confusing.

[jin]

As a data point, I think the --platform's flag values would surprise me the least if it's the lowest in precedence.

• Transitions: 1) explicit flag in transition, then 2) flag from platform.
• Command line: 1) explicit flag in command line, 2) bazelrc, 3) flag from platform.

Accumulation of flags should also be in that order IMO.

[aiuto]

+1 to Jin's proposal. With maybe an exception that setting a flag on the command line and then setting --platform where that changes the flag could either fail or give a warning.

One thing for sure. Having both the flag explicit and in the platform has to have the same relative precedence in a transition or from the command line.

[gregestren]

See #18672 (comment):

AFAIU Bazel intentionally inlines --config=foo so explicit command-line flags don't necessarily override bazelrc-set ones.

[jin]

cc @Colecf

This is great!

[sluongng]

First of all, I think this is a great suggestion.

In general, I have been thinking about something like

# BUILD

flag_group(
  name = "a",
  include = {
    "common": { "foo": "bar" },
    "build": { "key": "val" },
  },
)

in hope to get more structure than .bazelrc today.

So here are my questions:

1. What kind of flags are supported? build, common? what if it's a test or coverage or query exclusive flag?

2. Is it possible to set --platform=B inside platform(name="A")? What happens then?

3. (perhaps not related) Do we have a way to automatically set the platform based on the host environment (i.e. from a repository_rule)? It would help user detects special hardware easier. For example: "hey I have a GPU so let me use GPU-specific platform".

[fmeum]

I haven't gotten around to writing the doc for making the auto-configured host platform extensible, but I can do it anytime when it becomes relevant for other designs - it's not a complicated design at all.

[illicitonion]

@fmeum I'd love to see that doc - I have some hacks I'd be very happy to delete currently customising the default host platform!

[gregestren]

Out of scope for this proposal, but I'm a bit worried we'll eventually set ourselves up for a trap over which flags apply where.

As @fmeum and @katre note , we're starting to explore the idea of named flag groups in BUILD files. I had some ideas around that, although I'm not sure now if we want yet another place to define flag groups if this current proposal effectively fulfills that.

Implementation aside, the broad idea will naturally encourage users to define meaningful blobs of flags: combinations that have some higher-level meaning than each flag does, referenced by a name that clearly expresses that meaning. This is a great thing: it makes it easier for users to express their needs from their point of view, not the lower-level build API's point of view. i.e. making build a higher-level language.

This will naturally encourage users to associate flag groups with targets, reducing the need to set flags at the command line. So we move away from patterns like

$ bazel build //:foo --flagA=1 --flagB=2 ... 

to just:

$ bazel build //:foo

or even better:

$ bazel build //:all

Even better! But now we run into the problem that bazelrc files cover more flags than transitions, or this proposal, or config_settings can. This isn't a shallow difference: bazelrcs apply at a very different part in the code base. There are some hard limitations that would make it impossible for these other mechanisms to match bazelrc's scope.

This might subvert the $ bazel build //:all ideal. We end up where some flags fit within that ideal, but other flags have to be set in a bazelrc. So we still have $ bazel build //:all --config=foo. And it's not clear to users which flags go where or why. And we lose flexibility to configure different targets in //:all differently. And it's just not as clean of an outcome.

I think we can mitigate this. But it's on my mind.

[katre]

One important point (which I should explicitly note in the proposal) is that, just like with transitions, we can't possibly support --config= in platform-based flags because that's essentially a macro phase in flag parsing, and doesn't exist any more once Bazel starts.

Ideally if we add a future flag_group rule/concept, we can migrate platform-based-flags to platform-based-flag-groups and use the same abstraction.

[aiuto]

Not that this addresses Greg's concern, but the way I think about what we set in --config (or the command line) vs. platform flags is that platform flags are there to precisely specify the needs of the targets hardware/OS combination, while there will always be a need for independent, user defined flags that specify specific application variations within that platform.

This might help in our examples and thoughts on it. In my world, no one should ever set CPU or microarchitectures in a bazelrc or a flag_group. That is the domain of platform flags.
Conversely, no one should set --define=lang=klingon in a platform flag. That is a feature of this application instance.

I would go as far as to say that perhaps we should ban --define from platform flags. If you can use a define, you probably should have a constraint_value instead.

[sluongng]

An example with inheritance would look like:

Could you please be a bit more specific on how flags inheritance will result in the end?
Are we simply appending the list/array or should entries be deduplicated by flag key?

[katre]

My thought was that this, at least, can be handled the same way that we currently handle command line options: any flag in the child platform will end up either overriding or add to the same flag from a parent platform.

Having the same flag repeated in the platform would be the same, just handled in order down the list.

[Colecf]

Have we considered restricting this to just build settings instead of arbitrary flags? I still think it would be better to move to a world where transitioning on flags is really deprecated.

[katre]

One goal here is to remove the need for the platform-to-flag portion of the platform mappings file. A large part of that is making sure that a valid --cpu (and frequently --crosstool_top) is set for each platform defined. Because of that, we need to allow at least some native flags in this list.

Why do you think it would be helpful to restrict to only Starlark flags?

[Colecf]

I thought we were trying to replace --cpu with constraint settings/values. Platform mappings files are explicitly a tool for easing the transition off of deprecated behavior, is that also the goal of this proposal? I thought this was supposed to be a long-term "what we really want" kind of feature.

I don't have too strong opinions either way, because I think this whole proposal is a half-step anyways compared to what I proposed here.

[katre]

Replacing --cpu is a long-term goal, but as a migration path for Bazel users platform mappings are too centralized to work well. We also want this feature for longer-term changes.

[aiuto]

Accumulating flags are an interesting problem. Let's say we have a rule that transitions to platform A, and then transitively depends on something that transitions to A again. Does the no-op change of platform do nothing?
It can't really, because if the path as transition to A, ... rule that transitions a single flag from A ... rule that transitions to A, then we do want to reset that flag. At least I believe that should be the behaviors.

Now, lets say A has --copt = [-X]. What I don't want is for the second transition to A to result in --copt = [-X,. -X]. That would potentially be a different configuration that the user didn't intend.

[gregestren]

This sounds like a case where named / statically defined / non-patch configurations would fit.

If the configuration is explicitly defined then it doesn't matter what flags were in previous configurations.

[pcjanzen]

Accumulating flags are an interesting problem even without no-op transitions. Let's say we have --platforms=//platform:foo on the command line, then we have multiple platform transitions in series to //platform:bar and //platform:baz.
Clearly we don't want the targets that are configured for //platform:baz to have the flags from foo or bar.

It seems like in order to make this work sensibly, we'd have to maintain a special bucket of platform-added flags, and unwind the entire old platform flag bucket each time the platform changes.

(Naively I would think that the platform flag bucket would override the command line options, but I haven't thought too deeply about it).

[gregestren]

Same response as my comment above - I have not-yet-shared proposals exploring the idea of configurations that don't depend on the parent configurations. If we can make that model work I think it would clean up these ambiguities nicely.

[aiuto]

Clearly we don't want the targets that are configured for //platform:baz to have the flags from foo or bar.

Probably, but if another transition in the middle reset one of the flags from bar, what would people expect? I can imaging a transition that "developer hardware" and then a second one to "android_arm64". Composition may be better than reset. Resetting back to the last transition, would add a lot of mechanism that users would still not understand if people don't define their platforms well and use transitions appropriately.

Along with this proposal, we should have examples of what makes sense in platform flags vs. what is an abuse of it. I don't want to spend a lot of time getting "right" behavior for things which people hack up but could be done better. Here are some examples.

1. prefer a constraint over a --define whenever possible.

Why do people use --define? Usually to have a config_setting based off of that and then use it to select() behavior. Just use the constraint. We can explicitly state that having --define=OS=LINUX in the platform is the wrong way to do it.

2. copts is good example of the accumulatiing flags problem you cand end up with sets of -Dvar that are not what one would ever set from the command line. Don't do it. Use a constraint and then extend your toolchains to trigger the needs flags based on that constraint. (We have ideas on making that easier than it sounds).

The more I write on this, the more inclined I am to say we should just ban accumulating flags for now. There is always a way to not use them.

[gregestren]

Probably, but if another transition in the middle reset one of the flags from bar, what would people expect?

My argument with named configs is that you don't have to reason about the propagating consequences of transitions since they don't propagate beyond their immediate result.

Lots of exceptions to that principle. But I think that paradigm shift makes a lot of unintuitive nuances go away.

Stated differently: less reliance on trying to understand implicit dynamically created configurations that don't exist in any user-accessible API like a BUILD file or invocation command.

[gregestren]

Acknowledging reviewer request.

My main feedback is the same as in the other proposal: I really want to sit down and explore more concretely how this fits with the other new concepts we're exploring: where they overlap, where they could conflict, what they do uniquely, how they'd fit most elegantly. I need some more time to get momentum on that (such as sharing my named config / flag set proposal here) so we can broadly contextualize the conversation.

I think the right thing is that I whip up that proposal and share and get that thread going. Then we write a holistic "how everything fits together" doc that references all these proposals. That doc would rationalize the past, present, and future of the combination of: config_setting, constraint_setting, constraint_value, build_setting, platform, toolchain, target_compatible_with, flag groups / named configs, change_platform, split_platforms, platform_data, and other things I'm forgetting.

Each is a puzzle piece and we want to be as absolutely clear as possible as what the completed puzzle looks like. As a bonus we might get a refreshed long-term vision doc for free. :)

I think this is also healthy for the community to see, since it's really hard to see the forest through the trees when we're looking at individual focused proposals.

[gregestren]

Update: still want to do this. But practicality & time mean that's going to have to follow up as a later concern.

[katre]

After talking with @gregestren and looking at my calendar, I'm going to pause the discussion on this proposal: @gregestren has a related proposal to finish up and I will be out of the country for most of July.

I'd like to pick this back up when I'm back in the last week of July.

[katre]

I'm back, and reviewing the new comments and responding to the old.

[sdtwigg]

I have some concerns, especially from the implementation complexity side where we will likely have to start making finer distinctions on when things happen in options parsing and handling compared to before. platform_mappings, for better or worse, sidestepped these concerns because it was highest priority and non-overriddable. (Not that this was good; just to explain why this has not come up before.)

I think it would be reasonable to first consider what happens during the initial options parse and ignore the potential impact of transitions. Notably, only here do we have to consider blazerc and the actual commandline arguments.

Will platforms-based flags have higher or lower priority than blazerc settings? (I think regardless we will have to add a new entry to OptionPriority.PriorityCategory to get this to work cleanly and allow us to more seamlessly tune the ordering behavior.)

If choosing the behavior where the order of setting --platforms matters, would it then behave more like config expansion (where the priority of the set flags after expansion inherits the priority of the pre-expansion flag)? (Note that I think this would require redoing parts of the OptionParser implementation to effectively add a more general version of --config expansion.)

Will the flags attribute of platforms be non-configurable? (It essentially has to be in order for the feature to be sanely usable as part of the initial options parse but I did not see a mention in the proposal.)

Can platforms itself be set with platform-based flags? (This sounds silly but isn't inconceivable as users may try to use platforms+platform-based-flags to define a transition and config-in-one.)

Now, considering transitions (but no longer blazerc or commandline arguments):

Will the platform-based flags only be applied once, when platforms is transitioned? (I presume they are not applied 'continuously', like with platform_mappings as then they would effectively always have highest priority.)

What if there are multiple platforms defined in platforms? (This is really just to push for platforms becoming single-valued)

What if the transitions sets other flags, besides just platforms? (set platform from transition -> apply platform-based-flags -> apply all other flags from transition -> apply platform_mappings)

What if the transition setting the flag is the exec transition? (Wherein it is conceivable that you want the platform-based flags to take precedence)

What if the transition explicitly sets the platform to the same value? (Should platform-based flags be re-applied even though, technically, the platform change was seemingly a no-op.)

What if the transition clears the platform and thus it is platform_mappings setting the platform? (We may need to double apply here so that the platform is determined correctly: apply all flags from transition -> determine platform via platform_mappings -> apply platform-based-flags -> apply all non-platform flags from transition -> apply platform_mappings)

[sdtwigg]

As an aside, I have a mild concern as to how this interacts with attempts to replace CPU with platforms in the output directory name. Namely, I fear that giving platforms more expansive features will lead to users creating more of them unnecessarily (e.g. instead of just an "x86" platform they would have "x86-but-with-Java12", etc.) and this will not play nice with having to give each platform unique shortnames.

(Amusingly, having the platform-based flags include platform kind of subverts this as then the x86-but-with-Java12 never really exists as a platform but instead exists only as a way to jointly define a config+transition. But also this just seems like a hack on top of a hack.)

[jin]

As an aside, I have a mild concern as to how this interacts with attempts to replace CPU with platforms in the output directory name.

For Android, we have hundreds of different target platforms. There's no simple "x86" platform - it's "Pixel 6 arm64 userdebuggable" or "Virtual Android x86_64 eng-build" or "Reference Hardware ABC123 RISCV". Consider the combinatoric aspect of that. Each product platform in turn expands to hundreds of little build settings. We cannot embed the hundreds of build settings into the platform name, but platform based flags would let us design that abstraction pretty well.

@Colecf

[katre]

Many useful comments, thanks @sdtwigg!

I think it would be reasonable to first consider what happens during the initial options parse and ignore the potential impact of transitions. Notably, only here do we have to consider blazerc and the actual commandline arguments.

Will platforms-based flags have higher or lower priority than blazerc settings? (I think regardless we will have to add a new entry to OptionPriority.PriorityCategory to get this to work cleanly and allow us to more seamlessly tune the ordering behavior.)

The initial flag parse is definitely the hard part here.

I had envisioned this as happening like platform_mappings is handled currently: as the last stage, the platform is read and any flags are then applied, effectively being the highest priority and even overwriting the command line.

I agree that this is not optimal. The best approach would be to effectively expand the platform-based flags at the same level as the actual --platforms flag (acting similar to config, as you note) (and specifically ignoring the effective default --platforms=@local_config_platform//:host).

The problem here is that we'd only want to expand the last version of --platforms, so that an over-ridden --platforms from a bazelrc can't leave "ghost flags" in the configuration.

This is a pretty confusing set of semantics, and also doesn't mesh well with the current flag parsing system. Is there a cleaner way to formalize this?

Will the flags attribute of platforms be non-configurable? (It essentially has to be in order for the feature to be sanely usable as part of the initial options parse but I did not see a mention in the proposal.)

Yes, this should absolutely be non-configurable, I'll clarify that in the proposal.

Can platforms itself be set with platform-based flags? (This sounds silly but isn't inconceivable as users may try to use platforms+platform-based-flags to define a transition and config-in-one.)

This should also be disallowed. Ideally we'd somehow disallow "platform-like" flags such as --android_platforms, but that may be too tricky to formalize.

Now, considering transitions (but no longer blazerc or commandline arguments):

The flags should only be applied if the value of --platforms changes during a transition (whether the exec transition or another transition, including native ones like AndroidSplitTransition.

What if there are multiple platforms defined in platforms? (This is really just to push for platforms becoming single-valued)

This is one of the biggest YAGNI mistakes in the platforms API. I'd love to unwind it and make --platforms single valued. Is that worth doing as part of this proposal? Note that currently it's an warning to pass multiple values (see `PlatformConfiguration.reportInvalidOptions).

What if the transitions sets other flags, besides just platforms? (set platform from transition -> apply platform-based-flags -> apply all other flags from transition -> apply platform_mappings)

My immediate thought is that this is the right order: any flags set specifically in the transition override flags from the platform.

Also, we should disable platform mappings when using platform-based flags. I'll clarify both of these in the proposal.

What if the transition explicitly sets the platform to the same value? (Should platform-based flags be re-applied even though, technically, the platform change was seemingly a no-op.)

I think that no, the flags should not be re-applied (I'll happily listen to arguments to the contrary).

What if the transition clears the platform and thus it is platform_mappings setting the platform? (We may need to double apply here so that the platform is determined correctly: apply all flags from transition -> determine platform via platform_mappings -> apply platform-based-flags -> apply all non-platform flags from transition -> apply platform_mappings)

Ugh, that's complicated but makes sense. Adding to the proposal.

output directory name

What's the current plan for handling this? An attribute on the platform or a global mapping? In either case, if the project declares that two distinct platforms have the same output directory, can we rely on them to be correct enough (or on Bazel to give useful errors)?

[katre]

The proposal now has a Semantics section based on @sdtwigg's suggestions. Thanks all!

[gregestren]

Thoughts:

Syntax:

I don't fully understand the syntax concerns. The proposal describes:

    flags = [
        "--custom_malloc=//label",
        "--other_flag",
        "--//example/starlark:flag=23",
    ],

This is clearly intuitive. It matches what the user's used to at the command line. But we have two other BUILD constructs that also itemize flags:

def _transition_impl(settings, attr):
    _ignore = (settings, attr)
    return {
        "//command_line_option:cpu": "arm",
        "//some:starlark_flag": "foo",
    }

config_setting(
    name = "flag_blob",
    values = {
        "//command_line_option:cpu": "arm",
        "//some:starlark_flag": "foo",
    })

Never mind the bazelrc syntax.

Is this further forking the syntax, if the current norm people are used to in BUILD files is {}?

Re: Rejected Alternative: Starlark Dict type, the transition logic works with mixed build types. But I think that's not translatable from pure .bzl code to a rule attribute like you're doing here. So I think that objection stands.

Re: Parsing already exists, doesn't config_setting already solve that? I acknowledge config_setting gets weird with allowMultiple flags.

The main reason I'm bringing this up is the principle of consistent experience. If the reasons are good enough to diverge here that's fair. But let's be super-conscientious so users down the road don't complain "Why does Bazel do the same thing in two different ways?"

Other:

• Possibly other platform-like flags (such as --android_platforms) should also be disallowed?

  My inclination is to not worry about that as it's a language-specific concern (we can't definitively lock these down).

• If platform-based flags are used, platform mapping will be skipped

  Very cool. Sets it up nicely for clean migration.

• Ideally, the flags from the selected target platform should have the same priority as the --platforms flag itself.

  This mostly makes sense. But if the ordering of precedence is explicit flags override bazelrc-set flags, doesn't that mean $ bazel build //foo --platforms=//:explicit_platform --config=foo prioritizes --platforms=//explicit_platform's settings over anything in --config=foo?

  I actually don't remember if this is current bazelrc semantics. Or if --config=foo is filled in "in-place" in its position in the explicit flag list.

• About the ordering of --config=foo (where the --platforms flag is the middle entry in the --config): that makes sense but I'm also worried this could be hard to implement. Flag parsing has to be changed to proactively look for the --platforms flag, run separate logic to extract out its flags, then inject them back into the in-process parsing. I'm sure it's possible but if that's different parts of the Bazel code base it could get messy. I guess it largely comes down to how easily we can access Target instances from the parsing logic. Maybe ConfigSetting.java has precedent.

• Re: the transition implementation, that might require updating the transition cache to include Platform targets?

• For native transitions, which instead operate directly on BuildOptionsView instances, processing will be more difficult

  This is only for native transitions that set --platforms? How many are there?

• we may need to change the ~30 native transitions to use an API more similar to Starlark transitions

  I'd think this could be really hard. Maybe one approach is to pipe the platform-set flags into a custom Starlark transition then compose that with the native transition, then let Bazel's machinery automatically handle the composed logic.

• Transitions which unset the value of --platforms ...

I didn't catch how the double application fits? I saw #18672 (comment) but didn't fully follow.

• Nit: why are there two "Semantics" sections?

[katre]

Re: Syntax (ie, --flag vs //command_line_option:flag)

I think that //command_line_option: is a misfeature: it exists because the initial implementation of transitions worked on Starlark transitions, so it assumed the key of the dict was a label, and then command line flags had to be coerced to a label. This has led to user confusion in the past, because nothing else understands the //command_line_option: pseudo-package. (Ask me separately about my opinions on other pseudo-packages like //visibility and //conditions).

Transitions at least have a better solution for types, because bzl files can use String: Object dictionaries, so that flag values can use the appropriate type instead of having the values as strings. This isn't possible currently in BUILD files: rule attributes can only use a few dict types, none have Object values, and in the past the responsible team has stated that they do not want to add more dict types.

I'm confused about your comment on config_setting: the config_setting rule has different attributes for native and Starlark flags, so your eample would be:

config_setting(
    name = "flag_blob",
    values = {
        "cpu": "arm",
    },
    flag_values = {
        "//some:starlark_flag": "foo",
    },
)

Given this, I think the least confusing is to use the same flag syntax in the new platform attribute as users would use on the command line. Ideally someday we can fix transitions to use the same.

Precedence

But if the ordering of precedence is explicit flags override bazelrc-set flags, doesn't that mean $ bazel build //foo --platforms=//:explicit_platform --config=foo prioritizes --platforms=//explicit_platform's settings over anything in --config=foo?

I believe the obvious approach here would be that, if the same flag is present in //:explicit_platforms and config:foo, then the later one (in this case, from the config) would override the former. If the flag has allow_multiple set, then they would obviously accumulate.

I'm not 100% sure that this is actually possible to implement with current flag parsing. If it isn't, then the platform would override, and Bazel should emit a warning that this is the case.

Native Transitions

This is only for native transitions that set --platforms? How many are there?

Good question. Looking at the google-internal repository, I see around 5, but I may be missing transitions which change --cpu (and thus implicitly change the platform via platform mapping). So it's not a big number to handle specially.

Unsetting The Platforms Flag

The problem with this case is that when --platforms is cleared, it's because the transition wants platform mapping to determine the new target platform. However, platform mapping to find a platform works by checking all set flags, so first we need to fully apply the transition (to learn new flags), then run platform mapping (to learn the new platform), then apply the flags from the platform.

If the platform then changes a flag from the original transition, this is a confusing state, and the assumption would be that the transition's flag would be set (since it is, in some sense, the higher priority).

Another option is to declare that if a platform is set by platform mapping, it's flags always take precedence. It might be possible to also check this case, and when a platform is set by platform mapping, if the flags are in both the transition and the platform (and don't have the same value), declare that is an error.

Two Semantics Sections

Bad edits on my part, I am fixing this.

[gregestren]

From https://bazel.build/run/bazelrc#config:

--config=foo expands to the options defined in the rc files "in-place" so that the options specified for the config have the same precedence that the --config=foo option had.

That means for

$ bazel build //foo --platforms=//:explicit_platform --config=foo

we'd have to first scan to find the --config=foo setting for config:foo --platforms=//:foo_platform and ensure we don't apply the explicit --platforms=//:explicit_platform's flags?

[katre]

Yes, and this seems both confusing and difficult. As I said earlier, if it's not possible we should at least detect and warn.

[gregestren]

I think that //command_line_option: is a misfeature...

Oh, I didn't mean to advocate for forced label syntax for native flag names. Happy to drop that as not in consideration here.

I'm confused about your comment on config_setting: the config_setting rule has different attributes for native and Starlark flags, so your eample would be:

config_setting(
    name = "flag_blob",
    values = {
        "cpu": "arm",
    },
    flag_values = {
        "//some:starlark_flag": "foo",
    },
)

You're right, I got the example wrong.

Maybe we should consider the reverse. Move config_setting or even transition() to:

config_setting(
    name = "flag_blob",
    values = [
       "--cpu=arm",
       "--//some:starlark_flag=foo"
    ]
)

Given this, I think the least confusing is to use the same flag syntax in the new platform attribute as users would use on the command line. Ideally someday we can fix transitions to use the same.

Ah yes, just as you stated. I'd really love to not need so many attributes on config_setting.

Anyway. my main point is for us to stay conscientious of consistent syntax. It makes sense to properly focus on that when/if we get to #18672 (comment). So no more in scope for this conversation.

But if the ordering of precedence is explicit flags override bazelrc-set flags, doesn't that mean $ bazel build //foo --platforms=//:explicit_platform --config=foo prioritizes --platforms=//explicit_platform's settings over anything in --config=foo?

I believe the obvious approach here would be that, if the same flag is present in //:explicit_platforms and config:foo, then the later one (in this case, from the config) would override the former. If the flag has allow_multiple set, then they would obviously accumulate.

I'm not 100% sure that this is actually possible to implement with current flag parsing. If it isn't, then the platform would override, and Bazel should emit a warning that this is the case.

Got it. For reference the below comment (#18672 (comment)) clarifies the prioritization logic.

Unsetting The Platforms Flag

The problem with this case is that when --platforms is cleared, it's because the transition wants platform mapping to determine the new target platform. However, platform mapping to find a platform works by checking all set flags, so first we need to fully apply the transition (to learn new flags), then run platform mapping (to learn the new platform), then apply the flags from the platform.

If the platform then changes a flag from the original transition, this is a confusing state, and the assumption would be that the transition's flag would be set (since it is, in some sense, the higher priority).

Makes sense, thanks.

Another option is to declare that if a platform is set by platform mapping, it's flags always take precedence. It might be possible to also check this case, and when a platform is set by platform mapping, if the flags are in both the transition and the platform (and don't have the same value), declare that is an error.

What happens today if a transition sets --foo=a --bar=b --platforms=P and that triggers a platform mapping of P: --foo=c --bar=d?

Isn't the current semantics that the platform mapping exclusively applies after the transition? Which already raises the possibility of conflicts?

This all sounds pretty hypothetical: the exact mix of transitions, flags, platforms-with-flags, and platform mappings meant for some effect. If the above is right, why don't we just keep it simple: transition first, then platform-map, then we're done. That would still require steps 1-4 but not 5. Slightly simpler implementation and IMO simpler API contract. i don't think we know for a fact yet that fails actual user expectations? And if it did we could always adjust?

We could go even farther and decide to not yet support transitions that clear --platforms. That completely avoids platform-based-flags and platform mappings co-existing, which is a worthy goal that drives this proposal. Then worry about that as an incremental adjustment. Do you know any specific cases we'd need this for?

[katre]

Right now, the platform mapping is always at highest priority (since it is applied after everything else). We can certainly make the same choice for platform-based flags, but it's very inflexible and has led to issues in the past. On the other hand it's much simpler, easier to understand, and we can revisit if we have problems later. I'll defer to you and @sdtwigg on this.

Clearing the --platforms flag to force toolchain resolution happens now in several native and Starlark transitions, to force platform mapping to match the flags set by a rule that does not directly support platforms. I don't think this is a feature we can get rid of until we remove platform mapping entirely.

[gregestren]

On the other hand it's much simpler, easier to understand, and we can revisit if we have problems later.

This approach still makes sense to me. I don't yet sense the implications of missing this optimization are dire, or unknowable before a major milestone like a Bazel release.

[gregestren]

LGTM on the proposal after consensus on this.

[katre]

I've updated the design to reflect this (see bazelbuild/proposals#341).

[gregestren]

Thanks. To be super-clear: LGTM, as-is, from this reviewer. (@sdtwigg is the other reviewer).

[sdtwigg]

LGTM frome me

[katre]

Thanks for the comments, everyone! Now that this is accepted, I am locking the discussion to keep it archived, but feel free to reach out or file further feature requests if needed.

[katre]

Update on platform-based flags

I have an implementation mostly ready to start submitting. Unfortunately, the original planned semantics aren't feasible to implement.

What we wanted

Ideally, platform-based flags would have fixed a number of problems with platform mappings, the biggest being that platform mappings are effectively highest priority, it's not possible to override the flags from the mapping to a different value. The original ideal for platform-based flags was that they would only be applied when the target platform changes, thus allowing flags to be overridden, and that we'd augment the flag parsing so that the values could be more flexible.

What is possible

Unfortunately, those semantics aren't possible at this time without a major overhaul of Bazel's flag parsing library (specifically, we'd need to merge the currently split functionality in OptionsParser and StarlarkOptionsParser). This is very dangerous, as the flag parsing is very complex and, historically, it's been hard to make changes without breaking other features (which is why that split exists today).

What we can get

Instead, the current version of platform-based flags that I plan to submit will have the same semantics as platform mappings: the flags from the platform will be applied to every configuration, and will not be able to be overridden by other flags (either in transitions or at the command line). This is unfortunate, but will still allow us to use platform-based flags in a reduced manner. Hopefully at some point we can come back and add the missing functionality to the option parsing and improve this.

Please feel free to comment or message me to discuss this further if you have questions.

[gregestren]

To be clear, if:

• platform A sets flagB=c
• we build //foo:target
• and //foo:target depends on //foo:dep
• and //foo:dep applies transtion(flagB=d)

that transition will be overridden and have no effect?

i.e. any target built for a platform takes its settings even if it's not actively transitioning from another platform?

Which part of the OptionsParser / StarlarkOptionsParser split creates problems?

[katre]

any target built for a platform takes its settings even if it's not actively transitioning from another platform

That's correct.

Which part of the OptionsParser / StarlarkOptionsParser split creates problems?

OptionsParser is what handles --platforms (and thus would need to load in a target to read the flags attribute), but StarlarkOptionsParser is what is able to read build targets and use the data in options parsing.

Handling transitions is a slightly different problem but is mostly a matter of checking for platform changes and doing accounting.

[gregestren]

Can we avoid the OptionsParser / StarlarkOptionsParser non-integration by handling this outside options parsing?

For example, Starlark transitions today have pre-execution native logic that loads all input Starlark flags, i.e. inputs = ["//foo:my_starlark_flag"]. I think that loading logic is necessary because //foo:my_starlark_flag might not have been loaded yet from the top-level configuration. So transitions need to load it to know what default value to feed into their implementation code. I think that logic is something like returning a null value to tell Skyframe to evaluate those targets.

Can the same logic back-out the transition, load the platform, get its flags, then implicitly add them to the transition's outputs? Or construct a new transition around them so the whole things becomes a ComposedTransition?

Not sure how any of the above applies for --platform=foo at the top-level.

I'm also still not sure what semantics we ultimately want. I know there's discussion upthread and in the original proposal. But I"m really not sure what the ideal combination is of:

• $ bazel build //foo --platforms=a --flag=b
• transition(foo) sets --platform=a
• //foo is built with --platforms=b, depends on //bar, //bar applies transition(bar) with no platform change
• A platform switch happens with no new info about what to do with the flags. Should all platform flags reset on changing platforms? Should some of them be permanent?
• How any of this applies with aggregating flags like --copt.

This reminds me of the related theme of an intelligent flag propagation model. We're already thinking about that re: project boundaries. Maybe this should be part of the story.

[gregestren]

Also, what's the key difference to this vs. platform mappings today. Especially if we're inhering a lot of the platform mapping semantics. I think:

• Different intention: platform mappings are for associating platforms with their equivalent pre-platform --cpu, etc. settings. Platform-based flags are for setting flags that are intrinsically associated with a platform, and naturally complement it
• Less surprise behavior since the settings are defined clearly in the platform definition instead of some global file you may not know about
• More "automatic" since you doin't have to register a mapping file in your depot. Take any platform definition and use it as is on any repo.

I hope we can remove as much implementation complexity as possible with this shift, as platform mapping were really tough on the Skyframe evaluation model. I understand you've already refactored to improve that story.

[katre]

The parser issues don't apply to transitions, so we could in theory make this work for transitions, but I think that's even more confusing. We'll get a lot of support requests along the lines of "I can do this in a transition but when I try to use the same flags at the command line I get different effects".

Trying to handle options parsing after the actual parser (which is what StarlarkOptionsParser is effectively doing, it delays handling all Starlark options) is tricky: after parsing is done we've lost all information about flag precedence: this means that there's no difference between --platforms=foo --flag_from_the_platform and --flag_from_the_platform --platforms=foo. However, with configs from rcfiles, we've taught users that these are different. Again, I see a lot of potential support issues from this logic.

[tpudlik]

Thanks for the update.

the flags from the platform will be applied to every configuration, and will not be able to be overridden by other flags (either in transitions or at the command line)

What happens if you transition to a platform that specifies different values for the same flags?

Or will it not be possible at all to transition to a platform that specifies flag values? (That would be truly unfortunate, as it's the major way we envisioned using them!)

[katre]

The flags always are set based on the target platform. If you transition platforms, that platform's flags will be set.

Flags set by the previous flag will not be removed, they will still be present (unless overridden by the new platform's flags or another transition).

[tpudlik]

That's great. I think the inability to override the flags from the commandline is slightly inconvenient but not a big deal. It does reinforce the case for what I believe was already recommended best practice, i.e. that platforms should be defined in "leaf" projects (not in library repositories that will be reused by other projects). Since any flags set in an "upstream" platform would be impossible to override.

The inability to override flags from transitions is a potential headache, but I don't think it would matter for us in practice today.

Incidentally, how do platform-based flags interact with platform inheritance? Are they inherited, but can be overridden?

[katre]

Incidentally, how do platform-based flags interact with platform inheritance? Are they inherited, but can be overridden?

Correct, they'll use the same system as execution property inheritance.

The only nuance is that there's no easy way to say "reset this flag to the default, whatever that was": if a parent platform sets a flag, the child platform can either accept that, or set it to a different value, but can't just remove it.