-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using watches and handling timeouts #92
Comments
I have not seen |
Changing the timeout does not help, it is the Kube API server which terminates the connection. |
@kke any ideas? |
Why not just catch K8s-client could of course handle that automatically internally... |
If this is still an issue, feel free to reopen / report again. |
Hi, I am playing with the watch feature. I got it working for my use case but the code quits automatically after a while. What is the best way of keeping it alive? Thanks |
[brain dump, not sure all relevant for k8s-client, HTH] There are several scenarios for restarting watches.
I suspect a client lib could automatically retry from last seen version, but when that's too old it better surface the error to caller ? Compare python client discussion kubernetes-client/python#972, kubernetes-client/python-base#133. |
Hi @cben ! How do I restart from the last seen resourceVersion? At the moment I am first saving the list of the existing resources (Velero backups) in an array, and then when the watch starts I ignore those backups that already existed. But it sounds like it would be much better if I could restart the watcher from where it left. How can I do this with this gem? Thanks! |
OK I see that I can set the resourceVersion as parameter for the watch method. But what value do I specify initially, so that I don't just get the whole list of the existing resources? |
@vitobotta Not really sure what you are trying to achieve, but usually these are handled with something like:
So yes, initially when your app starts, you need to get all the resources through watch. If you are "syncing" the status with something external, your app needs to decide what to do in case the resource has been already seen in the past and possibly exists in the external thingy. |
I ended up doing a list first, getting the max resource version and using that. Seems to work. Thanks! :) |
Small correction: If doing a list, you should use the whole list's kubernetes devs are pretty adamant that But your instinct was a good one, pointing to a subtle distinction between list and watch:
|
Hi @cben ! How do I get the resourceVersion for the list? I tried with |
Found it! It's |
Yes, that's the one I meant. |
I've been using https://github.com/abonas/kubeclient library, but I like the flavor of this library better.
I'm a heavy user of the watch API, and when testing out this code:
After roughly 5 minutes the exception handler reports:
ERROR -- : Watcher error: end of file reached (EOFError)
I sort of expected this, if you issue
kubectl namespaces -w
it will timeout in about the sameamount of time.
We are stuck on an older version of Kubernetes (1.89) and the Kube API server was getting hammered because the abonas client was not handling the closing of the terminated connection and it was causing the API server to backup and affect the cluster.
What is the proper way to handle the timeout/EOFError ?
The text was updated successfully, but these errors were encountered: