-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remediate intermittent deploy failures/timeout on prometheus #122
Comments
Just encountered this myself, the istio sidecar isn't terminating correctly on the kube-prometheus patch something or other, job container is done but sidecar won't die |
Yeah that's the assumption. Pepr is supposed to terminate that job. It's an edge case, but not totally sure if it's the watch getting dropped or something in the actual job termination code. |
I do want to say after like 8 minutes the sidecar did eventually terminate, so its like pepr is getting hung up on something |
I'm running into this as well in the |
Should be resolved by #419 - leaving open until that is released and we have feedback though. |
Tentatively closing - please reopen or create a new issue if you encounter this problem in latest core versions. |
Describe what should be investigated or refactored
Occasionally in local dev and frequently in CI Prometheus is failing to deploy successfully and deployment times out without manual intervention.
Additional context
This may be related to pepr istio container job termination and the admission/patch jobs not getting killed but we dont always have good feedback as to if this is the case in ci.
The text was updated successfully, but these errors were encountered: