-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/prune-contaiers: add a GC script for containers images #3826
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,168 @@ | ||
#!/bin/bash | ||
set -euo pipefail | ||
|
||
# Prune containers from a remote registry | ||
# according to the images age | ||
|
||
print_help() { | ||
cat 1>&2 <<'EOF' | ||
Usage: prune-containers [OPTIONS] REGISTRY_URL | ||
|
||
Prune old FCOS images from remote container registry. | ||
|
||
Example: prune-containers --devel 3w quay.io/fedora/fedora-coreos | ||
|
||
The following options are available: | ||
|
||
-a Path of the authentication file. Passed directly to skopeo. | ||
-d Maximum age of images from the development streams. | ||
If not specified, no images will be deleted. | ||
See durations section below for the accepted arguments. | ||
-p Maximum age of images from de production streams. | ||
See durations section below for the accepted arguments. | ||
If not specified, no images will be deleted. | ||
-z Dry run: do not delete any image, just print what action would be done. | ||
-h Print this help | ||
|
||
Age syntax: age of the images can be specified in years, months, days or weeks. | ||
Use Y/y for years, M/m for months, D/d for days, W/w for weeks. | ||
|
||
EOF | ||
} | ||
|
||
convert_to_seconds() { | ||
local input=$1 | ||
local duration=${input:0:-1} | ||
local unit=${input: -1} | ||
local seconds=0 | ||
|
||
case $unit in | ||
"y" | "Y") | ||
seconds=$(( duration * 365 * 24 * 60 * 60 )) | ||
;; | ||
"m" | "M") | ||
seconds=$(( duration * 30 * 24 * 60 * 60 )) | ||
;; | ||
"w" | "W") | ||
seconds=$(( duration * 7 * 24 * 60 * 60 )) | ||
;; | ||
"d" | "D") | ||
seconds=$(( duration * 24 * 60 * 60 )) | ||
;; | ||
*) | ||
echo "Invalid unit. Please use y (years), m (months), w (weeks), or d (days)." | ||
exit 1 | ||
;; | ||
esac | ||
|
||
echo $seconds | ||
} | ||
usage="Usage: prune-containers [-hz] [-a path] [-d devel_duration] [-p prod_duration] REGISTRY_URL" | ||
|
||
REPOSITORY_URL="" | ||
PROD_AGE=0 | ||
prod_age_seconds=$(date +%s) | ||
DEVEL_AGE=0 | ||
devel_age_seconds=$(date +%s) | ||
SKOPEO_ARGS="" | ||
DRY_RUN=false | ||
|
||
while getopts 'hza:d:p:' options; do | ||
case "$options" in | ||
h) | ||
print_help | ||
exit 1 | ||
;; | ||
a) | ||
SKOPEO_ARGS+="--authfile $OPTARG " | ||
;; | ||
d) | ||
DEVEL_AGE=$OPTARG | ||
devel_age_seconds=convert_to_seconds $OPTARG | ||
;; | ||
p) | ||
PROD_AGE=$OPTARG | ||
echo $OPTARG | ||
prod_age_seconds=convert_to_seconds $OPTARG | ||
;; | ||
z) | ||
DRY_RUN=true | ||
;; | ||
?) | ||
echo "error: unrecognized option: $VALUE" | ||
echo $usage | ||
exit 2 | ||
;; | ||
esac | ||
done | ||
shift "$((OPTIND -1))" | ||
|
||
if [ "$#" -ne 1 ]; then | ||
echo "Error: REGISTRY_URL argument is required." | ||
echo $usage | ||
exit 1 | ||
fi | ||
|
||
REPOSITORY_URL=$1 | ||
|
||
skopeo_delete() { | ||
|
||
image=${REPOSITORY_URL}:$1 | ||
if $DRY_RUN; then | ||
echo "Dry-run mode, would delete ${image}." | ||
else | ||
skopeo delete ${SKOPEO_ARGS} docker://${image} | ||
fi | ||
|
||
} | ||
|
||
echo "Pulling tag list from ${REPOSITORY_URL}" | ||
# This is a JSON object: | ||
# {"Repository": "quay.io/jbtrystramtestimages/fcos", | ||
# "Tags": [ | ||
# "40.20"40.20240301.1.0",.....]} | ||
tags_json=$(skopeo list-tags docker://"${REPOSITORY_URL}") | ||
|
||
tags=$(jq -r '.Tags[]' <<< "${tags_json}") | ||
|
||
# To compare the dates we convert them into seconds since epoch | ||
today=$(date +%s) | ||
|
||
for tag in $tags; do | ||
|
||
# ignore the named moving tags ("stable", "next" etc..) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I would generalize this as "ignore all tags that don't match the build ID format". |
||
if [[ "$tag" == "stable" || "$tag" == "next" || "$tag" == "testing" \ | ||
|| "$tag" == "rawhide" || "$tag" == "testing-devel" \ | ||
|| "$tag" == "next-devel" ||"$tag" == "branched" ]]; then | ||
echo "skipping $tag tag." | ||
continue | ||
fi | ||
|
||
# split the build id into an array | ||
IFS='.' read -r -a build_id <<< "${tag}" | ||
build_date=${build_id[1]} | ||
stream=${build_id[2]} | ||
|
||
# assign the number of seconds between the build date | ||
# and today. Then we directly compare with the policy | ||
# value | ||
tag_age=$(( today - $(date -d "${build_date}" +%s) )) | ||
|
||
# test for production streams. Production stream id are 1, 2 and 3 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it'd be nice to follow a model closer to |
||
if [[ $stream -eq 1 || $stream -eq 2 || $stream -eq 3 ]]; then | ||
if [[ $tag_age -gt "$prod_age_seconds" ]]; then | ||
echo "Production tag ${tag} is older than ${PROD_AGE}, pruning." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This isn't considering barrier releases. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess i would have to pull the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The source of truth for barriers is in https://github.com/coreos/fedora-coreos-streams/tree/main/updates. You can download from there, or from the S3-uploaded version (e.g. https://builds.coreos.fedoraproject.org/updates/testing.json). |
||
skopeo_delete "$tag" | ||
fi | ||
fi | ||
|
||
# test for devel streams | ||
if [[ $stream -eq 10 || $stream -eq 20 || $stream -eq 91 || $stream -eq 92 ]]; then | ||
if [[ $tag_age -gt "$devel_age_seconds" ]]; then | ||
echo "Devel tag ${tag} is older than ${DEVEL_AGE}, pruning." | ||
skopeo_delete "$tag" | ||
fi | ||
fi | ||
|
||
done | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm surprised by the choice of bash. I think any new command with non-trivial logic should be written in one of the other two languages in this repo (Python or Go).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comments below are only about overall logic and not bash-specific things.