Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report remote execution messages as events #18757

Closed

Conversation

illicitonion
Copy link
Contributor

Currently the message is appended to a spawn's stderr, which pollutes that output. It also means that whether a message is output is gated by the --output_filter flag, which is surprising - metadata messages don't feel like "output" in the same way --output_filter applies to.

Instead, emit the message as a top-level message to the terminal.

Currently the message is appended to a spawn's stderr, which pollutes
that output. It also means that whether a message is output is gated by
the `--output_filter` flag, which is surprising - metadata messages
don't feel like "output" in the same way `--output_filter` applies to.

Instead, emit the message as a top-level message to the terminal.
@illicitonion illicitonion requested a review from a team as a code owner June 23, 2023 14:25
@github-actions github-actions bot added awaiting-review PR is awaiting review from an assigned reviewer team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Jun 23, 2023
@coeuvre
Copy link
Member

coeuvre commented Jun 23, 2023

cc @exoson who originally implemented this feature.

@coeuvre coeuvre added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jun 23, 2023
@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jun 26, 2023
@brentleyjones
Copy link
Contributor

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 26, 2023
@exoson
Copy link
Contributor

exoson commented Jun 26, 2023

I think I'll need these in the build event stream such that I can easily get the messages for a specific build/test target. With this change, the messages will only be available somewhere within the full log in the build events, right? Otherwise this change seems to be a really good one.

@coeuvre
Copy link
Member

coeuvre commented Jun 26, 2023

With this change, the messages will only be available somewhere within the full log in the build events, right?

If I am not mistaken, it's still in the progress message.

@exoson
Copy link
Contributor

exoson commented Jun 26, 2023

Yea, that's my understanding too. My worry is that it doesn't seem to be trivial to map the re messages from the progress messages to individual build/test targets.

@iancha1992
Copy link
Member

iancha1992 commented Jun 26, 2023

@bazel-io fork 6.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 26, 2023
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Jun 26, 2023
Currently the message is appended to a spawn's stderr, which pollutes that output. It also means that whether a message is output is gated by the `--output_filter` flag, which is surprising - metadata messages don't feel like "output" in the same way `--output_filter` applies to.

Instead, emit the message as a top-level message to the terminal.

Closes bazelbuild#18757.

PiperOrigin-RevId: 543365674
Change-Id: I9874c8a0946a3156a2c17a2962880184d3aeebe0
@coeuvre
Copy link
Member

coeuvre commented Jun 27, 2023

My worry is that it doesn't seem to be trivial to map the re messages from the progress messages to individual build/test targets.

A workaround is to let server embed the label/action id/whatever in the message. Or Bazel could prepend the label to the message.

@illicitonion illicitonion deleted the re-message-event branch June 27, 2023 09:45
iancha1992 added a commit that referenced this pull request Jun 27, 2023
Currently the message is appended to a spawn's stderr, which pollutes that output. It also means that whether a message is output is gated by the `--output_filter` flag, which is surprising - metadata messages don't feel like "output" in the same way `--output_filter` applies to.

Instead, emit the message as a top-level message to the terminal.

Closes #18757.

PiperOrigin-RevId: 543365674
Change-Id: I9874c8a0946a3156a2c17a2962880184d3aeebe0

Co-authored-by: Daniel Wagner-Hall <dwagnerhall@apple.com>
@EdSchouten
Copy link
Contributor

EdSchouten commented Aug 24, 2023

Is anything being done to address @exoson 's request? It is unpleasant that this data is no longer part of the Build Event Streams in a somewhat structured way.

Out of curiosity, why did this get merged into 6.3.0, even after concerns were raised? Wouldn't it have been better if consensus was reached first?

@coeuvre
Copy link
Member

coeuvre commented Aug 24, 2023

Is anything being done to address @exoson 's request? It is unpleasant that this data is no longer part of the Build Event Streams in a somewhat structured way.

Nope. I am not sure what is missing for the server to display these messages in a structured way given that it already has the target label.

Out of curiosity, why did this get merged into 6.3.0, even after concerns were raised? Wouldn't it have been better if consensus was reached first?

I thought not including the message in spawn's stderr is more correct because technically they are not spawn's output. I didn't reject the cherry-pick into 6.3 because the change looked innocent to me. I still believe the concern can be addressed on server side. I am sorry if this change causes trouble for you.

If you think it's better to address that from Bazel side, please fire an issue and we can discuss there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants