Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious errors in candidate scheduler logs #138

Open
adrienjt opened this issue Feb 15, 2022 · 0 comments
Open

Spurious errors in candidate scheduler logs #138

adrienjt opened this issue Feb 15, 2022 · 0 comments

Comments

@adrienjt
Copy link
Contributor

If a candidate pod is not allowed (e.g., another one was chosen, in a different target cluster), by design, it fails the PreBind candidate scheduler plugin by timeout. Unfortunately, this generates errors in the scheduler's logs, which are misleading, e.g.

E0215 19:45:37.807953       1 framework.go:967] "Failed running PreBind plugin" err="timed out waiting for the condition" plugin="candidate" pod="default/follow-7564df7669-2v5n6-2f456"
E0215 19:45:37.808232       1 factory.go:225] "Error scheduling pod; retrying" err="running PreBind plugin \"candidate\": timed out waiting for the condition" pod="default/follow-7564df7669-2v5n6-2f456"
I0215 19:45:37.808304       1 factory.go:231] "Pod doesn't exist in informer cache" pod="default/follow-7564df7669-2v5n6-2f456" err="pod \"follow-7564df7669-2v5n6-2f456\" not found"
E0215 19:45:37.815203       1 scheduler.go:322] "Error updating pod" err="pods \"follow-7564df7669-2v5n6-2f456\" not found" pod="default/follow-7564df7669-2v5n6-2f456"

Note: regarding the two last lines, by timeout, the candidate pod actually doesn't exist anymore, because it's been deleted by the proxy scheduler.

We should find a way to silently abort PreBind.

@adrienjt adrienjt changed the title Spurious errors in candidate scheduler Spurious errors in candidate scheduler logs Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant