Skip to content

Commit

Permalink
executor: suppress spurious log messages (#11273)
Browse files Browse the repository at this point in the history
Suppress stats streaming error log messages when task finishes.
Streaming errors are expected when a task finishes and they aren't
actionable to users.

Also, note that the task runner Stats hook retries collecting stats
after a delay. If the connection terminates prematurely, it will be
retried, and closing the stats stream is not very disruptive.

Ideally, executor terminates cleanly when task exits, but that's a more
substantial change that may require changing the executor/drivers interface.

Fixes #10814
  • Loading branch information
Mahmood Ali committed Oct 6, 2021
1 parent 5624603 commit bc2a51d
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 6 deletions.
3 changes: 3 additions & 0 deletions .changelog/11273.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:bug
client: Removed spurious error log messages when tasks complete
```
16 changes: 10 additions & 6 deletions drivers/shared/executor/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ import (
"github.com/hashicorp/nomad/helper/pluginutils/grpcutils"
"github.com/hashicorp/nomad/plugins/drivers"
dproto "github.com/hashicorp/nomad/plugins/drivers/proto"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)

var _ Executor = (*grpcExecutorClient)(nil)
Expand Down Expand Up @@ -132,12 +134,14 @@ func (c *grpcExecutorClient) handleStats(ctx context.Context, stream proto.Execu
return
}

if err != nil {
if err != io.EOF {
c.logger.Error("error receiving stream from Stats executor RPC, closing stream", "error", err)
}

// End stream
if err == io.EOF ||
status.Code(err) == codes.Unavailable ||
status.Code(err) == codes.Canceled ||
err == context.Canceled {
c.logger.Trace("executor Stats stream closed", "msg", err)
return
} else if err != nil {
c.logger.Warn("failed to receive Stats executor RPC stream, closing stream", "error", err)
return
}

Expand Down

0 comments on commit bc2a51d

Please sign in to comment.