command for garbage-collect

[ghost]

Hello,

What is the command for the garbage-collect?

Does it execute on the host?

Thank you!

[Joxit]

Hello,
The command is registry garbage-collect config.yml (It's displayed when you delete an image).

Yes you must execute this on the host (or in the container of your private registry)

[ghost]

I have an idea for an ENV variable on the Registry container.

• SET ENV Var for Garbage-Collect_CRON=True

This will enable garbage collection ON set to a default of every 24hrs

I would edit the code myself but I just cant find it. ;-)

[Joxit]

Sorry, but the registry image is out of the scope of this project.
The registry image is managed by the docker team here (docker/distribution)

[ghost]

Thank you..

In the interim I am running on HOST:

Dry Run..

sudo docker exec ui-as-proxy_registry_1 registry garbage-collect --dry-run /etc/docker/registry/config.yml

OR - Live..

sudo docker exec ui-as-proxy_registry_1 registry garbage-collect /etc/docker/registry/config.yml

I will try to make an entrypoint file or see if CMD will run properly at end of service config in compose file..

[ghost]

@Joxit ,

After a delete and garbage collect the orphaned images are deleted but there is a listing in the Registry-UI that remains. When I click on it it shows no images within.

Is there a way to remove that orphaned listing from the UI as well?

[Joxit]

No, this is a limit from docker registry, see this #53 (comment)
You can delete manually old images by deleting folders.

[ghost]

Here is a snippit of a Bash script I am using for garbage collection and deletion of the orphaned repo..

get_array()
{
	curl --user ${REG_USER} http://${REGISTRY}/v2/_catalog >list
    sed -i -- s#repositories##g list;
    sed -i -- s#\{##g list;
    sed -i -- s#\}##g list;
    sed -i -- s#:##g list;
    sed -i -- s#\\[##g list;
    sed -i -- s#\\]##g list;
    sed -i -- s#\"##g list;
    sed -i -- s#\"##g list;
    sed -i -- s#,#" "#g list;
	ARRAY1=$(cat list)
	garbage_collect
}

garbage_collect()
{
	docker exec ui-as-proxy_registry_1 registry garbage-collect /etc/docker/registry/config.yml	&&
	del_repo 
}

del_repo()
{  
        for i in ${ARRAY1[@]}; do 
    	curl --user ${REG_USER} http://${REGISTRY}/v2/$i/tags/list | grep null ;
      	if [ $? -eq 0 ]; then
        docker exec -it ui-as-proxy_registry_1 rm -rf /var/lib/registry/docker/registry/v2/repositories/$i
    	fi
  	done
}

Note: after deleting an image via the Repository UI the repository remains and is empty - orphaned if you will.. Creating a bash script with the snippit of code to iterate through the tags works to garbage collect and delete the orphaned repository via tags found.

To test properly I needed to restart the registry and UI containers then push the same image back into the registry container - otherwise the layer still exists for the old image somehow and would not allow me to push the image over. After restart of containers i was then able to push my image back into the registry and the UI would show the image once again...

Thank you! I hope this helps others..

//Edited for a fix...

[Herr-Sepp]

Thanks for sharing your script.

I could not use the get_array() part so i have achieved the same with grep and substituion.
I also delete the repositoryfolder if the registry dont know the repository any more but it is still listed insed the catalog (NAME_UNKNOWN).

I leave it here if anyone want to use this:

repositories=( $(curl -s http://${REGISTRY}/v2/_catalog | grep -o '"[^"]*"') )

docker exec registry2 registry garbage-collect /etc/docker/registry/config.yml

for i in ${repositories[@]//\"/}; do
		curl -s http://${REGISTRY}/v2/${i}/tags/list | egrep '"tags":null|NAME_UNKNOWN';
		if [ $? -eq 0 ]; then
		docker exec -it registry2 rm -rf /var/lib/registry/docker/registry/v2/repositories/$i
		echo "delete empty repository $i"
		fi
done

[honoref]

without curl command, it's possible to check empty folder with a find -exec ls -ld. If the ls return only 2 inodes, than the folder is empty (no -empty flag for the find command of the registry image :-( )

To find all tags folder's without any tags :
find /var/lib/registry/ -type d -iname "tags" -exec ls -ld {} \;|grep "[ ][ ]2[ ]1000[ ][ ][ ][ ][ ]1000[ ]"

grep <number_of_inodes> <user_id> <group_id>. In registry container, user and group equal 1000.
Be careful about spaces in grep filter, markdown in this post trunk spaces

Extract only the image name folder with a sed, all before the time (time include) and remove subfolders /_manifests/tags
sed "s/.* [0-9][0-9]:[0-9][0-9] //;s/\/_manifests\/tags$//"

Exec find and remove folders inside the container :
docker exec registry rm -Rf $(docker exec registry find /var/lib/registry -type d -iname "tags" -exec ls -ld {} \;|grep "[ ][ ]2[ ]1000[ ][ ][ ][ ][ ]1000"|sed "s/.* [0-9][0-9]:[0-9][0-9] //;s/\/_manifests\/tags$//")

After that, perhaps some repositories haven't any images. To find theses repositories :
find /var/lib/registry/docker/registry/v2/repositories/ -type d -maxdepth 1 -mindepth 1
same grep, but for the sed, only need to remove all before the time
docker exec registry rm -Rf $(docker exec registry find /var/lib/registry/docker/registry/v2/repositories/ -type d -maxdepth 1 -mindepth 1 -exec ls -ld {} \;|grep "[ ][ ]2[ ]1000[ ][ ][ ][ ][ ]1000[ ]"|sed "s/.* [0-9][0-9]:[0-9][0-9] //")

[mrashad10]

• I have written this script

#!/bin/sh

deleted=$(registry garbage-collect /etc/docker/registry/config.yml 2>/dev/null |grep "eligible for deletion$"|cut -d' ' -f4)
if [ "$deleted" -eq 0 ];then exit;fi

cd /var/lib/registry/docker/registry/v2/repositories/
for d in $(ls -d *);do
    count=$(ls -1 $d/_manifests/tags/|wc -l)
    if [ "$count" -eq 0 ];then
       rm -fr $d
    fi
done

kill -15 1

• Save as file "cleanup.sh" within the same directory that I save docker registry data and certificates ...etc and mark as executable
• Mount that script using the -v switch like this "-v cleanup.sh:/cleanup.sh"
• Then call the cleanup script and if there any need it will restart the container like this "docker exec -it myregistry /cleanup.sh"

hope this help

[azadiali]

hi all
thank you for your scripts for delete dangling images...
but I'm confused :(
which script is complete?

[logopk]

I have noticed that some problems after deleting and garbage collection are from caching in the browser. Do a full refresh of the page to get updated listings.

My process is:

1. delete the tag $tag
2. delete the tag directory in the registry directory
   rm -rf /var/lib/registry/docker/registry/v2/repositories/$name/_manifests/tags/$tag
3. run garbage collection
   registry garbage-collect /etc/docker/registry/config.yml
4. force reload of the registry ui page

@azadiali: the scripts i have found messed up my registry! I had to recover from backup more than once...

[azadiali]

@Joxit
Hi
Did you try issue the "garbage-collect" command on host with LUA in nginx? is it possible?

[Yamakasi]

Has anyone something really working ? I have tried all examples but what I run against my docker registry, the UI still shows empty repos.

[logopk]

did you force-reload the page? In my case the json object was cached by the browser.

As I proxy the ui via an apache I set the no-cache header and that solved the problem.

[Yamakasi]

@logopk Yes I did, but my empty repo's are still there.

Can you exactly post what and how you run things at this current moment ?

[logopk]

Can you exactly post what and how you run things at this current moment ?

Apache (tls, proxy) > UI > registry:2.8.2

what do you mean with "exactly"?

I'm not happy with the garbage collector. It destroyed my registry more than once. So I usually backup docker volumes and return to the backup if sth. goes wrong.

[Yamakasi]

@logopk the gabrage collector is not nice at all, I wondered if you used some extra script for it.

[hmoffatt]

@mrashad10 nice script thanks but it doesn't work if you have registries with / in their name.

Anyway I think you could run the cleanup on the host instead of insider the container, then you have a better find.

[logopk]

To whom it may concern:

there is a serious bug/non-feature in the garbage collection with multi arch images. That was causing my delete issues and destroyed registry...

distribution/distribution#3178

With this in consideration, deleting in the ui and running garbage-collection on the server fixes my issues.

[littletuna4]

Does anyone know why this isn't run automatically everytime an image is deleted? It feels like it should be the default behavior to me?

[Joxit]

Hi @littletuna4, I agree you should ask distribution/distribution
There are a lot of issues about garbage collection 😅