Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle eraser pod and PodTemplate deletion on controller manager restart #912

Open
ashnamehrotra opened this issue Nov 13, 2023 · 1 comment
Assignees
Labels
bug Something isn't working
Milestone

Comments

@ashnamehrotra
Copy link
Contributor

ashnamehrotra commented Nov 13, 2023

What steps did you take and what happened:
[A clear and concise description of what the bug is.]
AKS ImageCleaner user raised issue:
If eraser-controller-manager pod is restarted, the ImageJob is cleaned up but the PodTemplate and eraser pods are left behind (since controller-manager-pod is not deleted in this case). In this example, manager pod restarted twice and two PodTemplates are left behind:

...
eraser-aks-upools1z3-40582422-vmss00003t-7wsdb 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003t-d5hbv 0/3 Completed 0 2d20h
eraser-aks-upools1z3-40582422-vmss00003u-ckw2s 0/3 Completed 0 2d20h
eraser-aks-upools1z3-40582422-vmss00003u-x8sg7 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003v-2hgsf 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003v-5wbp6 0/3 Completed 0 2d20h
eraser-controller-manager-7bcc57b846-n52f7 1/1 Running 2 (43h ago) 3d20h

...


kubectl get imagejob -A
No resources found


kubectl get podtemplate -A
NAMESPACE NAME CONTAINERS IMAGES POD LABELS
kube-system imagejob-dhtbn collector,remover,trivy-scanner mcr.microsoft.com/oss/eraser/collector:v1.2.2,mcr.microsoft.com/oss/eraser/remover:v1.2.2,mcr.microsoft.com/oss/eraser/eraser-trivy-scanner:v1.2.2
kube-system imagejob-r5vlc collector,remover,trivy-scanner mcr.microsoft.com/oss/eraser/collector:v1.2.2,mcr.microsoft.com/oss/eraser/remover:v1.2.2,mcr.microsoft.com/oss/eraser/eraser-trivy-scanner:v1.2.2

What did you expect to happen:
PodTemplates and Pods should be cleaned up if manager restarts, not just when it is deleted.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

  • Eraser version: v1.2.2
  • Kubernetes version: (use kubectl version):
@ashnamehrotra ashnamehrotra added the bug Something isn't working label Nov 13, 2023
@ashnamehrotra ashnamehrotra changed the title Handle pod and PodTemplate deletion on controller manager restart Handle eraser pod and PodTemplate deletion on controller manager restart Nov 13, 2023
@ashnamehrotra
Copy link
Contributor Author

cc @WilliamRockwellEvans

@ashnamehrotra ashnamehrotra self-assigned this Nov 29, 2023
@sozercan sozercan added this to the v1.4.0 milestone Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants