-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Store: If request ctx has an error we do not increment opsFailures counter #3179
Conversation
…unter Signed-off-by: Jarod Watkins <jarod@42lines.net>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we need to just skip context.Canceled
.
I think timeouts might be a legit failures, no?
pkg/objstore/objstore.go
Outdated
@@ -412,7 +412,7 @@ func (b *metricBucket) Delete(ctx context.Context, name string) error { | |||
|
|||
start := time.Now() | |||
if err := b.bkt.Delete(ctx, name); err != nil { | |||
if !b.isOpFailureExpected(err) { | |||
if !b.isOpFailureExpected(err) && ctx.Err() == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if !b.isOpFailureExpected(err) && ctx.Err() == nil { | |
if !b.isOpFailureExpected(err) && ctx.Err() != context.Canceled { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree timeouts are legit failures, but this comment apply to any place, not just here, right? Also, is there a good reason why we don't we the context.Canceled
check isOpFailureExpected()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or does Aborted != timeout?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree timeouts are legit failures, but this comment apply to any place, not just here, right? Also, is there a good reason why we don't we the
context.Canceled
checkisOpFailureExpected()
?
If that is a better place for this then I have no problems making the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isOpFailureExpected
is really only for user specified function what to expect. I believe Cancel is valid to be ignored no matter what user specifies, no? (:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right @bwplotka 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated the PR to use that context.
Is there a way for me to view the circleci results without giving them access to my Github account? |
You shold have acces to logs, no? https://circleci.com/gh/thanos-io/thanos/11705?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link |
When I attempt to visit that URL, I get redirected to login via Github/Bitbucket and it requires read/write access to both public and private repos. I just didn't know if there was a way to view the logs "ready only" or not having to login at all. |
Try incognito mode - works just fine (: |
Signed-off-by: Jarod Watkins <jarod@42lines.net>
Signed-off-by: Jarod Watkins <jarod@42lines.net>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing, thanks!
Some other solution at some point is to have error status as label (:
Good enough for now +1
Thank you both! |
Signed-off-by: Jarod Watkins jarod@42lines.net
Resolves #3149
Changes
If a request is cancelled or times out we will no longer increment the ops failure counter.