-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add trace message output #79
Conversation
…as been specified
Welcome @scaat! |
Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA. It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
/check-cla |
/assign @tallclair |
Some observations:
So I suggest (see below) to move to printing the trace to stdErr on This is however a change to how What do you all think? @kubernetes/klog-maintainers diff --git a/klog.go b/klog.go
index 17d2975..108e0b0 100644
--- a/klog.go
+++ b/klog.go
@@ -809,14 +809,13 @@ func (l *loggingT) output(s severity, buf *buffer, file string, line int, alsoTo
os.Exit(1)
}
// Dump all goroutine stacks before exiting.
- // First, make sure we see the trace for the current goroutine on standard error.
- // If -logtostderr has been specified, the loop below will do that anyway
- // as the first stack in the full dump.
- if !l.toStderr {
- os.Stderr.Write(stacks(false))
- }
- // Write the stack trace for all goroutines to the files.
trace := stacks(true)
+
+ // We make sure we see the trace for all goroutines on standard
+ // error in any case.
+ os.Stderr.Write(trace)
+
+ // Write the stack trace for all goroutines to the files.
logExitFunc = func(error) {} // If we get a write error, we'll still exit below.
for log := fatalLog; log >= infoLog; log-- {
if f := l.file[log]; f != nil { // Can be nil if -logtostderr is set.
`` |
Hello @hoegaarden |
I added the tests, so we have a harness to make deliberate changes to klog's behaviour. Specifically the printing of traces is a good example: It is not entirely clear when those are printed right now. I would like to see the traces on stderr either always or only if e.g. a flag But in general, when there is consensus on how klog should behave, we can just:
|
/assign @hoegaarden |
@scaat -- Thinking a bit more about this, I think we should print the traces whenever we print a fatal log message. This is the behaviour that is consistent with logging to files ( The only thing that I still don't really understand is |
@scaat: Cannot trigger testing until a trusted user reviews the PR and leaves an In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@hoegaarden Thanks. |
Any of the approvers wanna approve this? Or is there still something to sort out? |
@hoegaarden deferring to @DirectXMan12 for approve :) |
Was waiting until we have a release note listed in the PR description. |
@scaat Can you please add a release note into the PR description ... either the one I proposed or something else/clearer/better. @DirectXMan12 I don't think we have any tooling in place here in |
@hoegaarden I have already added the one you proposed into the PR description. What else do I need to do? |
@DirectXMan12 are we good to go from your perspective? |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
/assign @dims @DirectXMan12 /remove-lifecycle stale |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dims, hoegaarden, scaat The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This change made consumers of klog start outputting excessive amounts of stack trace logs when invoking a fatal error (xref kubernetes/kubernetes#107665 and kubernetes/kubernetes#94663) That seems like a breaking change, or at least one that reduces usability for existing consumers. |
@hoegaarden please see @liggitt 's concern above cc @serathius @pohly for additional thoughts |
I agree that this change reduces the usefulness of klog. In the default configuration the stack dumps go to stderr. If that is a console and there are many goroutines, the console scrollback buffer overflows an the actual error at the beginning of the dump gets lost. |
Let's revert this. To handle the case where all backtraces are wanted, I am proposing some new API to include that. See #316. |
This is a revert of kubernetes#79. Dumping the stack backtraces of all goroutines to stderr is not useful for processes which have many goroutines. The console buffer overflows and the original error which got dumped earlier is no longer visible. Even in CI systems where stderr is captured the full dump is often not useful. This was pointed out as a potential problem during the original PR review but only got more attention after the updated klog got merged into Kubernetes and developers there started to see the longer output.
After reverting kubernetes#79 users who want the full dump of all goroutines can still get it by calling github.com/go-logr/lib.Backtrace(lib.BacktraceAll(true)).
This is a revert of kubernetes#79. Dumping the stack backtraces of all goroutines to stderr is not useful for processes which have many goroutines. The console buffer overflows and the original error which got dumped earlier is no longer visible. Even in CI systems where stderr is captured the full dump is often not useful. This was pointed out as a potential problem during the original PR review but only got more attention after the updated klog got merged into Kubernetes and developers there started to see the longer output.
After reverting kubernetes#79 users who want the full dump of all goroutines can still get it by calling github.com/go-logr/lib.Backtrace(lib.BacktraceAll(true)).
After reverting kubernetes#79 users who want the full dump of all goroutines can still get it by calling github.com/go-logr/lib.Backtrace(lib.BacktraceAll(true)).
What this PR does / why we need it:
Add trace message output when severity is fatalLog and -logtostderr has been specified.
When -logtostderr is specified, I called klog.Fatal("fatal msg"), only
will be output on the console, and no trace information will be output on the console. But more often, I need that.
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)
format, will close the issue(s) when PR gets merged):Fixes #
Special notes for your reviewer:
Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
Release note: