Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconciliation for git source stuck with growing workqueue_unfinished_work_seconds #1432

Open
ndemeshchenko opened this issue Apr 4, 2024 · 1 comment

Comments

@ndemeshchenko
Copy link

Hi,
On my flux installation in AWS EKS, version

flux: v2.1.2
helm-controller: v0.36.2
image-automation-controller: v0.27.0
image-reflector-controller: v0.23.0
kustomize-controller: v1.1.1
notification-controller: v1.1.0
source-controller: v1.1.2

periodically I see source-controller being stuck and not completing/starting any reconciliation on git sources, while helm sources seems running fine.

I can't find anything in the source controller log during the time or before workqueue_unfinished_work_seconds starting to grow.

image

Any pointers on where to look to find a solution?
Or what information might help to dig deeper

@ndemeshchenko ndemeshchenko changed the title reconciliation stuck with growing workqueue_unfinished_work_seconds reconciliation for git source stuck with growing workqueue_unfinished_work_seconds Apr 5, 2024
@souleb
Copy link
Member

souleb commented Jun 20, 2024

Do you see anything in the controller logs? Please use --log-level=debug. Please also check your consumption metrics. It would also help to have a git repository description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants