Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm] Don't fail the CI job if Firefox tests fail in runtime-wasm #69615

Merged
merged 4 commits into from
May 23, 2022

Conversation

radical
Copy link
Member

@radical radical commented May 20, 2022

Debugger tests running on Firefox are known to be unstable right now. So, they can fail on unrelated PRs.
When they are run on runtime-staging, the test failures don't mark the whole job as failed.
But when they run on runtime-wasm, they do. And this shows up as failed in GitHub checks view, which is not helpful.

@ghost
Copy link

ghost commented May 20, 2022

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

.. since they are known to be unstable currently. Having the Firefox job be red because some tests failed is not useful, because it can happen on any PR that runs runtime-wasm pipeline, and is not useful information.

Author: radical
Assignees: -
Labels:

area-Infrastructure-mono

Milestone: -

@ghost
Copy link

ghost commented May 20, 2022

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

.. since they are known to be unstable currently. Having the Firefox job be red because some tests failed is not useful, because it can happen on any PR that runs runtime-wasm pipeline, and is not useful information.

Author: radical
Assignees: radical
Labels:

arch-wasm, area-Debugger-mono

Milestone: -

@radical
Copy link
Member Author

radical commented May 20, 2022

/azp run runtime-wasm

@azure-pipelines
Copy link

Azure Pipelines failed to run 1 pipeline(s).

@radical
Copy link
Member Author

radical commented May 20, 2022

/azp run runtime-wasm

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@radical
Copy link
Member Author

radical commented May 20, 2022

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@radical radical changed the title [wasm] Don't fail the job if Firefox tests fail [wasm] Don't fail the job if Firefox tests fail in runtime-wasm May 20, 2022
@radical radical requested review from akoeplinger and thaystg May 20, 2022 19:26
@radical radical marked this pull request as ready for review May 20, 2022 19:26
@radical radical requested review from vargaz and steveisok May 20, 2022 19:27
@radical radical changed the title [wasm] Don't fail the job if Firefox tests fail in runtime-wasm [wasm] Don't fail the CI job if Firefox tests fail in runtime-wasm May 20, 2022
@radical
Copy link
Member Author

radical commented May 23, 2022

/azp run runtime-wasm,runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines failed to run 2 pipeline(s).

This reverts commit 849760c.
@radical
Copy link
Member Author

radical commented May 23, 2022

/azp run runtime-wasm,runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@radical
Copy link
Member Author

radical commented May 23, 2022

The unrelated AOT build failure is likely #69653 .
I cancelled the manual pipelines, once I confirmed that the fix still works.

@radical radical merged commit 110b4cf into dotnet:main May 23, 2022
@radical radical deleted the ff-no-fail branch May 23, 2022 23:22
@jakobbotsch
Copy link
Member

@radical I'm seeing libraries test runs show up as "!" in AzDO on failures and being reported as succeeded on GitHub. For example, my change at #68703 has many failures in "runtime-coreclr libraries-pgo", yet it does not show up as failed on GitHub. Seems related to this change?

@radical
Copy link
Member Author

radical commented May 24, 2022

I'm looking at that right now. If I don't figure out a solution soon, then I'll revert it.

@radical
Copy link
Member Author

radical commented May 24, 2022

@jakobbotsch #69736

@ghost ghost locked as resolved and limited conversation to collaborators Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture area-Debugger-mono
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants