-
Notifications
You must be signed in to change notification settings - Fork 17.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
testing: print log output immediately when it can #23213
Comments
But only with |
@cznic Absolutely - I'm not proposing any change in whether the log output is shown, just when it appears. |
As noted in #23567 (a duplicate of this), the issue also affects |
This is also a problem if the test crashes. I noticed this when testing go code with a buggy OpenCL driver. Having the log output flush after each call would make it much easier to find the exact point in the test that failed. |
Sounds like there is no investigation required here - the proposed change is clear. Labelling as NeedsDecision. /cc @mpvl |
I'll add my 2c here quickly: The fact that -v does not show logs means I get no sense of timing in failed test runs. Nor do I see failed tests, as @nerdralph mentioned. It's a real bummer to mass replace t.Log with fmt.Println, get my tests fixed, and then mass replace fmt.Println with t.Log. |
@ianlancetaylor - Can we get this in for 1.11 ? What are the factors that are yet to be "decided" (because of the needsdecision label) ? I have got bitten by this several times and would be great not to resort to printing through fmt package. Also curious regarding why it was initially designed not to print any output until the test has finished. Printing output immediately after a call to |
The current output is:
We can't start printing log output without knowing whether to start with PASS or FAIL. Various things read this output, we can't just reorder it. |
Ah, got it. |
Ah, I'd missed that detail. Would it be possible to to add some kind of a flag to enable this somehow, even if it involves changing the output syntax in that case? Something like |
@rsc What about adding a The lack of log output for long-running tests is a big problem, as it makes it harder to debug (especially when things go wrong, such as a test hanging or when you have race conditions), and some CI systems, such as CircleCI, terminate the tests if there has been no log output for too long (e.g., 10 min). I understand the need for backwards compatibility in the test logs, but I'm sure we can figure out a solution that solves both problems, rather than just closing the issue, and telling everyone to do a massive search & replace every time they want to debug... |
I believe that if you |
Hm, that's not what I'm seeing: func TestFoo(t *testing.T) {
t.Parallel()
for i := 0; i < 5; i++ {
t.Logf("%d", i)
time.Sleep(3 * time.Second)
}
}
func TestBar(t *testing.T) {
t.Parallel()
for i := 0; i < 5; i++ {
t.Logf("%d", i)
time.Sleep(2 * time.Second)
}
}
func TestBaz(t *testing.T) {
t.Parallel()
for i := 0; i < 5; i++ {
t.Logf("%d", i)
time.Sleep(1 * time.Second)
}
} If I create a file with the 3 tests above and run |
Oh, I forgot to say: use |
That works for me, but as I mentioned in #24262 it's a real bummer that the workflow is "grep and replace t.Log with fmt.Printf, debug, grep and replace fmt.Printf with t.Log". |
@ianlancetaylor Right, but the central question in this issue is having |
The testing package framework is fundamentally poorly designed for long running tests that need to report progress as they go. I'm suggesting a workaround.
|
I appreciate the help. FWIW, we are using the workaround, which involves creating a custom logger and passing it all throughout the test code, but that's clunky. My hope is to discuss whether something can be done to make Go a better fit for this sort of testing. Happy to open a new issue if that's more appropriate!
I guess the key questions are (a) why is that the purpose of
It's awesome that Go has a lot of powerful testing tools built in. We tend to pass the |
So if
I find that |
Sure, we can discuss other approaches. I've seen one suggestion so far: add a |
Why do we not get |
What's the best way to do that? File a new bug in this repo? |
I just filed #24929 as a proposal for adding a |
what if in a test we call another func that use fmt.XXX to output the progress info if test can't flush out the output immediately? |
Diagnosing a test that hangs up is made harder because it's
T.Logf doesn't print any output until the test completes (which might
never happen).
Before a test has called t.Parallel, it would be nice if it could print
any log output immediately.
This would also make it easier to see test Logf output in sequential context with
print statements made by the code being called by the test.
This issue is made worse when logging infrastructure is hooked up
to T.Log so that it doesn't pollute normal test output by printing to
stdout.
The text was updated successfully, but these errors were encountered: