Skip to content
This repository has been archived by the owner on Mar 13, 2022. It is now read-only.

resource version reset default when event object resource version inv… #199

Closed
wants to merge 1 commit into from

Conversation

DrAuYueng
Copy link

Watch an API resource and stream the result back via a generator, when resource_version specified, may be cause "too old resource version" error, then result would be:
{ "type":"ERROR", "object":{ "kind":"Status", "apiVersion":"v1", "metadata":{ }, "status":"Failure", "message":"too old resource version: 38081900 (64845263)", "reason":"Gone", "code":410 } }
In this case, the func unmarshal_event can not parse the right resource version, actually is None,
the stream while loop continue, the next request will get ApiException: (500) error:
HTTP response headers: HTTPHeaderDict({'Content-Length': '186', 'Content-Type': 'application/json', 'Date': 'Fri, 19 Jun 2020 06:21:06 GMT'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"resourceVersion: Invalid value: \\"None\\": strconv.ParseUint: parsing \\"None\\": invalid syntax","code":500}\n'

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 20, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: DrAuYueng
To complete the pull request process, please assign yliaog
You can assign the PR to them by writing /assign @yliaog in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

Welcome @DrAuYueng!

It looks like this is your first PR to kubernetes-client/python-base 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-client/python-base has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jun 20, 2020
@DrAuYueng
Copy link
Author

@yliaog

@codecov-commenter
Copy link

Codecov Report

Merging #199 into master will increase coverage by 0.42%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #199      +/-   ##
==========================================
+ Coverage   92.23%   92.66%   +0.42%     
==========================================
  Files          13       13              
  Lines        1481     1554      +73     
==========================================
+ Hits         1366     1440      +74     
+ Misses        115      114       -1     
Impacted Files Coverage Δ
watch/watch.py 100.00% <100.00%> (ø)
config/kube_config.py 83.29% <0.00%> (-0.20%) ⬇️
config/kube_config_test.py 95.76% <0.00%> (+0.16%) ⬆️
config/incluster_config_test.py 97.84% <0.00%> (+0.55%) ⬆️
config/incluster_config.py 95.45% <0.00%> (+10.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b87a5fe...8b57234. Read the comment docs.

@DrAuYueng
Copy link
Author

kubernetes version:v1.14.6

@roycaihw
Copy link
Member

/cc

if self.resource_version is None:
# In some case like 'too old resource version',
# the event type would be ERROR, resource version will be None
self.resource_version = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Watching with resourceVersion=0 doesn't guarantee the client to continue watching from the last seen state, and therefore may cause missing watch events. The right way to handle a 410 Gone response is to perform a LIST request, and start the watch from the resourceVersion returned by that new list operation (see https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes for more details).

This is part of the problems kubernetes-client/python#868 was trying to address

Copy link
Author

@DrAuYueng DrAuYueng Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If catch event type of Error and continue like #133 ,will cause endless loop, current resourceVersion never change, always get Error of message:"too old resource version"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If catch event type of Error and continue like #133 ,will cause endless loop, current resourceVersion never change, always get Error of message:"too old resource version"

Make resourceVersion=0 make sure the watch stream still work even if the wrong init resource version set

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will cause endless loop, current resourceVersion never change, always get Error of message:"too old resource version"

I don't think that's what we want and it should not happen

perform a LIST request, and start the watch from the resourceVersion returned by that new list operation

the resourceVersion returned by a LIST request will be the latest resourceVersion, it won't be stale

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 20, 2020
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Nov 19, 2020
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@k8s-ci-robot
Copy link
Contributor

@fejta-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants