Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune images after successful Kubernetes upgrade #3231

Closed
judge-red opened this issue Jun 17, 2024 · 1 comment · Fixed by #3348
Closed

Prune images after successful Kubernetes upgrade #3231

judge-red opened this issue Jun 17, 2024 · 1 comment · Fixed by #3348
Assignees
Labels
customer-request kind/feature Categorizes issue or PR as related to a new feature. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.
Milestone

Comments

@judge-red
Copy link

Description of the feature you would like to add / User story

After ~2 Kubernetes version upgrades, we always receive alerts as disk usage has climbed >80% (of 50GB). We then address this by SSHing into all control plane nodes and executing sudo crictl rmi --prune to delete unused images, bringing us down to ~40% usage.

It would be nice if we could tell kubeone, probably with a flag, to prune the images on all control plane nodes after the control plane has been upgraded. It's not super elaborate to do it by hand, but it would be a quality of life improvement. The actual cleanup isn't even the "hard" task, but the alert that fires first is often something unrelated (e.g. node-exporter running out of disk space) and thus it first takes a bit of investigation.

With a --prune-images flag, we'd just add that to our pipeline job and wouldn't have to care about it anymore.

Solution details

Add a flag, such as --prune-images, to the kubeone CLI that runs sudo crictl rmi --prune on each control plane node after the upgrade (either on that node or of the whole cluster - you're probably the better judge of that). I guess ideally it could also be used to prune images even when no upgrade is performed, in case people want to run it separately after the upgrade was done.

Alternative approaches

SSH into each control plane node and execute the command manually.

Use cases

Additional information

@judge-red judge-red added kind/feature Categorizes issue or PR as related to a new feature. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management. labels Jun 17, 2024
@kron4eg kron4eg added this to the KubeOne 1.9 milestone Jun 17, 2024
@xmudrii xmudrii added priority/normal Not that urgent, but is important customer-request labels Jun 24, 2024
@kron4eg kron4eg assigned kron4eg and unassigned kron4eg Aug 6, 2024
@csengerszabo
Copy link
Contributor

/assign @mohamed-rafraf

@kron4eg kron4eg removed the priority/normal Not that urgent, but is important label Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-request kind/feature Categorizes issue or PR as related to a new feature. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants