Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

failing known-issues tests #6527

Closed
Fishrock123 opened this issue May 2, 2016 · 5 comments
Closed

failing known-issues tests #6527

Fishrock123 opened this issue May 2, 2016 · 5 comments
Labels
test Issues and PRs related to the tests.

Comments

@Fishrock123
Copy link
Contributor

There were no (visible?) known-issues ci jobs run yet so I ran one. there seems to be quite a few failures...

https://ci.nodejs.org/job/node-test-known-issues/3/

It is possible some of these are due to platform-specific tests, would there be a better way to denote those? Maybe produce tap skipped output?

cc @nodejs/testing @cjihrig
@Fishrock123 Fishrock123 added test Issues and PRs related to the tests. known issue test labels May 2, 2016
@cjihrig
Copy link
Contributor

cjihrig commented May 2, 2016

All of the failures are:

=== release test-stdout-buffer-flush-on-exit [negative] ===
Path: known_issues/test-stdout-buffer-flush-on-exit
Command: out/Release/node /home/iojs/build/workspace/node-test-known-issues/nodes/debian8-64/test/known_issues/test-stdout-buffer-flush-on-exit.js

Ping @Trott from 33c27f8#diff-bee0ad67db456c74881851e227542dbb. There doesn't seem to be a lot of consistency among platforms on this one.

@Trott
Copy link
Member

Trott commented May 3, 2016

Argh, looks like it's actually flaky. Sometimes it passes on a platform and sometimes it fails. Will tinker a bit in hopes of making it reliable...

@Trott
Copy link
Member

Trott commented May 3, 2016

I've got it failing-as-expected everywhere but Windows. There doesn't seem to be a way to use the known_issues.status file in this situation (where you are expecting failure) so I either need to hack test.py for it or else add some code to bail out of the test, but then someone has to remember to remove the bail out code when the test is moved to parallel.

@Trott
Copy link
Member

Trott commented May 4, 2016

Oh, SKIP seemed to get the job done. PR coming soon.

@Trott Trott mentioned this issue May 4, 2016
Closed
2 tasks
@orangemocha
Copy link
Contributor

In the status files there also used to be support for a FAIL status, though I am not sure if it was fully ported from the v8 test runner.

On May 4, 2016, at 02:03, Rich Trott <notifications@git.luolix.topmailto:notifications@github.com> wrote:

Oh, SKIP seemed to get the job done. PR coming soon.

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHubhttps://github.com//issues/6527#issuecomment-216703284

Trott added a commit to Trott/io.js that referenced this issue May 5, 2016
@Trott
test: fix unreliable known_issues test
e594ea5 
`test-stdout-buffer-flush-on-exit` was not failing reliably on POSIX
machines and not failing at all on Windows. Revised test fails reliably
on POSIX and is skipped (in CI) on Windows where the issue does not
exist.

Fixes: nodejs#6527
@Trott Trott closed this as completed in c4006dd May 6, 2016
evanlucas pushed a commit that referenced this issue May 17, 2016
@Trott @evanlucas
test: fix unreliable known_issues test
83dab80 
`test-stdout-buffer-flush-on-exit` was not failing reliably on POSIX
machines and not failing at all on Windows. Revised test fails reliably
on POSIX and is skipped (in CI) on Windows where the issue does not
exist.

Fixes: #6527
PR-URL: #6555
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joao Reis <reis@janeasystems.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this issue Jun 14, 2016
@Trott
test: fix unreliable known_issues test
c6d8b7c 
`test-stdout-buffer-flush-on-exit` was not failing reliably on POSIX
machines and not failing at all on Windows. Revised test fails reliably
on POSIX and is skipped (in CI) on Windows where the issue does not
exist.

Fixes: #6527
PR-URL: #6555
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joao Reis <reis@janeasystems.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this issue Jun 24, 2016
@Trott
test: fix unreliable known_issues test
be0bb5f 
`test-stdout-buffer-flush-on-exit` was not failing reliably on POSIX
machines and not failing at all on Windows. Revised test fails reliably
on POSIX and is skipped (in CI) on Windows where the issue does not
exist.

Fixes: #6527
PR-URL: #6555
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joao Reis <reis@janeasystems.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this issue Jun 24, 2016
@Trott
test: fix unreliable known_issues test
91e7b01 
`test-stdout-buffer-flush-on-exit` was not failing reliably on POSIX
machines and not failing at all on Windows. Revised test fails reliably
on POSIX and is skipped (in CI) on Windows where the issue does not
exist.

Fixes: #6527
PR-URL: #6555
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joao Reis <reis@janeasystems.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
test Issues and PRs related to the tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test: fix unreliable known_issues test Trott/io.js
4 participants
@Trott @Fishrock123 @orangemocha @cjihrig