test: mark test failing on AIX as flaky #8065
Conversation
|
I'm heading out on vacation so do you think once I get your ok I can land or do we need to wait the normal 48 hours. |
|
Could you use a |
We now have adequate AIX hardware to add AIX to the regular regression runs. However, there are a couple of failing tests even though AIX was green at one point. This PR marks those tests as flaky so that we can add AIX so that we can spot any new regressions without making the builds RED The tests are being worked under the following PRs - being worked under nodejs#7564 test-async-wrap-post-did-throw test-async-wrap-throw-from-callback test-crypto-random - being worked under nodejs#7973 test-stdio-closed - covered by nodejs#3796 test-debug-signal-cluster
|
don't know why I was thinking "doc:" instead of "test:" fixed |
|
LGTM. I think changes to .status files should be exempt from the 48-hour rule, but that's just my opinion. So I certainly won't complain if you land it sooner. Should definitely get a CI run first, though, as I'm not sure what types of mishaps can happen if there is a typo in the .status file. So...uh...CI: https://ci.nodejs.org/job/node-test-pull-request/3624/ |
Trott
changed the title
|
Thanks, agree on the CI run was going to do that, thanks for launching |
|
LGTM I agree with @Trott , marking tests as flaky should be able to land immediately. I started an AIX job with the exact same parameters that would have been used if the job had been launched from node-test-commit: https://ci.nodejs.org/view/All/job/node-test-commit-aix/331/ |
|
@joaocgreis thanks, I had run my report on onnode-test-commit-aix but good to see it as will will be landed. |
We now have adequate AIX hardware to add AIX to the regular regression runs. However, there are a couple of failing tests even though AIX was green at one point. This PR marks those tests as flaky so that we can add AIX so that we can spot any new regressions without making the builds RED The tests are being worked under the following PRs - being worked under #7564 test-async-wrap-post-did-throw test-async-wrap-throw-from-callback test-crypto-random - being worked under #7973 test-stdio-closed - covered by #3796 test-debug-signal-cluster PR-URL: #8065 Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
Landed as a0b3bc5 |
|
Added AIX back to node-test-commit Build on master to validate all is still green:https://ci.nodejs.org/job/node-test-commit/4519/ (had wrong link) |
|
This is strange. I thought test-aix was supposed to be yellow, not green. (Wrong setup in the status file maybe?) And even though it's green, it's showing up as red in the status posted to this PR. Huh? |
|
@joaocgreis I'm pretty sure I added the entries as suggested to make it yellow but do see its green instead. |
|
The status file seems good. I haven't seen anything yellow in a while in CI, I'll investigate this. The bits to show the status in PR's are quite new, and according to @jbergstroem still not final. I see a difference between what is in test-aix and test-linux, one runs if it succeeds and the other if it fails. But I'm no expert in this. |
We now have adequate AIX hardware to add AIX to the regular regression runs. However, there are a couple of failing tests even though AIX was green at one point. This PR marks those tests as flaky so that we can add AIX so that we can spot any new regressions without making the builds RED The tests are being worked under the following PRs - being worked under #7564 test-async-wrap-post-did-throw test-async-wrap-throw-from-callback test-crypto-random - being worked under #7973 test-stdio-closed - covered by #3796 test-debug-signal-cluster PR-URL: #8065 Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
So strange that the post-status-to-PR widget is showing test-aix failing even though it succeeds.... |
|
cc @jbergstroem ^ any ideas? |
|
Going to close this since PR landed. Open this issue to track nodejs/build#467 issue with how flaky test failures are reported. |
|
@Trott, @Fishrock123: yeah, the PR bot/jenkins stuff is not quite done hence my suggestion of delaying making it default. Fixed a few cases post this issue but not quite there yet! |
|
adding to lts watch but it is not landing cleanly. I'm going to wait until we run ci on staging to see if we have AIX failures and revisit |
Checklist
make -j4 test(UNIX), orvcbuild test nosign(Windows) passesAffected core subsystem(s)
test
Description of change
We now have adequate AIX hardware to add AIX to
the regular regression runs.
However, there are a couple of failing tests even
though AIX was green at one point. This PR
marks those tests as flaky so that we can add AIX
so that we can spot any new regressions without making
the builds RED
The tests are being worked under the following PRs
test-async-wrap-post-did-throw
test-async-wrap-throw-from-callback
test-crypto-random
test-stdio-closed
test-debug-signal-cluster