Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat SuperPMI failures in AzDO pipelines as pipeline failures #68196

Conversation

BruceForstall
Copy link
Member

Now, any SuperPMI failure in the replay, asmdiffs, and checked/release
asmdiffs pipeplines is ignored, and doesn't cause AzDO to treat the entire
job as failed. The job does get a "!" icon, but that's only if you visit the
AzDO UI: the GitHub page on a PR doesn't see that failure.

Now that we have more experience with these pipelines, and we are using
replay and asmdiffs with JIT PRs, we want the failures to be more visible,
and treated as failures.

Note that asm diffs should not be treated as failures.

Now, any SuperPMI failure in the replay, asmdiffs, and checked/release
asmdiffs pipeplines is ignored, and doesn't cause AzDO to treat the entire
job as failed. The job does get a "!" icon, but that's only if you visit the
AzDO UI: the GitHub page on a PR doesn't see that failure.

Now that we have more experience with these pipelines, and we are using
replay and asmdiffs with JIT PRs, we want the failures to be more visible,
and treated as failures.

Note that asm diffs should not be treated as failures.
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost assigned BruceForstall Apr 19, 2022
@BruceForstall BruceForstall added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Apr 19, 2022
@ghost
Copy link

ghost commented Apr 19, 2022

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

Now, any SuperPMI failure in the replay, asmdiffs, and checked/release
asmdiffs pipeplines is ignored, and doesn't cause AzDO to treat the entire
job as failed. The job does get a "!" icon, but that's only if you visit the
AzDO UI: the GitHub page on a PR doesn't see that failure.

Now that we have more experience with these pipelines, and we are using
replay and asmdiffs with JIT PRs, we want the failures to be more visible,
and treated as failures.

Note that asm diffs should not be treated as failures.

Author: BruceForstall
Assignees: BruceForstall
Labels:

area-CodeGen-coreclr

Milestone: -

@BruceForstall
Copy link
Member Author

@jakobbotsch @kunalspathak @dotnet/jit-contrib PTAL

Copy link
Member

@kunalspathak kunalspathak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that asm diffs should not be treated as failures.

However, I still see that you changed the asmdiff file as well? If there are asmdiffs, do you remember if the script's exit code is 0. If yes, then your change seems good. We will only show it failed when anything really fails during asmdiff.

@ghost ghost added the needs-author-action An issue or pull request that requires more info or actions from the author. label Apr 19, 2022
@BruceForstall
Copy link
Member Author

However, I still see that you changed the asmdiff file as well? If there are asmdiffs, do you remember if the script's exit code is 0. If yes, then your change seems good. We will only show it failed when anything really fails during asmdiff.

@kunalspathak superpmi.py already does the right thing: it returns 0 (success) on MISSING data, or if there are asm diffs. Actually, for chk/rel asm diffs, if there are asm diffs we do consider it a failure (return 1). This is because there should never be chk/rel asmdiffs, so that is a real failure we want a pipeline to report.

@ghost ghost removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Apr 19, 2022
Copy link
Member

@kunalspathak kunalspathak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BruceForstall BruceForstall merged commit de0b85f into dotnet:main Apr 19, 2022
@BruceForstall
Copy link
Member Author

Note: the spmi replays will fail because of an assert that was added with #68161 (until that is fixed).

@BruceForstall BruceForstall deleted the TreatSuperPmiFailuresAsPipelineFailures branch April 19, 2022 23:38
directhex pushed a commit to directhex/runtime that referenced this pull request Apr 21, 2022
…t#68196)

Now, any SuperPMI failure in the replay, asmdiffs, and checked/release
asmdiffs pipeplines is ignored, and doesn't cause AzDO to treat the entire
job as failed. The job does get a "!" icon, but that's only if you visit the
AzDO UI: the GitHub page on a PR doesn't see that failure.

Now that we have more experience with these pipelines, and we are using
replay and asmdiffs with JIT PRs, we want the failures to be more visible,
and treated as failures.

Note that asm diffs should not be treated as failures.
@ghost ghost locked as resolved and limited conversation to collaborators May 20, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants