mitigate test2json bug causing missing test output #102
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dnephin
force-pushed
the
print-missing-output
branch
from
74004d0
to
aa9ed6b
Compare
|
The problem with this workaround is that any failed subtest will cause the root test to print all the output for all subtests in the summary. I think it's going to require another check for the root test case to only add the extra files if none of the subtests have failed. |
dnephin
changed the title
dnephin
force-pushed
the
print-missing-output
branch
from
aa9ed6b
to
c2cbea1
Compare
|
I think I have resolved the above issue by including the changes from #104 and adding a |
In support of a workaround for golang/go#29755
Previously the summary would filter out all messages which include a '--- FAIL' or '--- SKIP'. With this change only the framing message for the test case will be removed. This change is in preparation for working around the test2json output misattribution bug, which will result in the summary including the output for all subtests in some cases. After that bug is fixed, and the workaround is removed, it may still be valuable to be more strict about this filtering.
dnephin
force-pushed
the
print-missing-output
branch
from
c2cbea1
to
6733193
Compare
See golang/go#29755. If a test fails, and there is no test output (other than framing), then print all the test output for the entire test case. This won't fix the issue if the test which failed had some output, but well behaved tests generally shouldn't non-failure message output, so hopefully this case is rare.
dnephin
force-pushed
the
print-missing-output
branch
from
6733193
to
cacb835
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See golang/go#29755, which may be fixed by golang/go#34419 if that is merged. Even if that fix makes it into go1.15 there will be plenty of projects still running earlier versions of Go, so a workaround may be useful for another year or more.
Related to #64, #66
If a test fails, and there is no test output (other than framing), then print all the test output for the entire test case.
This won't fix the issue if the test which failed had some output, but well behaved tests generally shouldn't non-failure message output, so hopefully this case is rare.