Problem with private registry on 17.06

[danieljuhl]

After upgrading docker to 17.06, shepherd no longer works, as updating a service requires the flag --with-registry-auth to be passed manually on each service update if the service was created with this flag.

I tried making a fork, and apply this flag for all service updates if engine is higher than 17.05, but though the command executed works if I'm connected directly to the docker node, it doesn't work as expected when executed by shepherd.

Could it be some missing permissions for the shepherd service?

[caoer]

probably need to pass the credential about the private registry info into docker environment.

I tried running ./shepherd on the host machine, the updates works.

[danieljuhl]

@caoer you are probably right.. how to pass/expose the credentials to shepherd? And how to ensure that shepherd has access to the credentials used during service create? Let's say I create a shepherd service, and then add new services to the swarm. I would then manually have to re-create the shepherd service, or?

[caoer]

to pass credential, use docker's environment virable, and then passed into the shepherd script.
I dont' think it needs manually re-create the service, if manually...this script just makes no sense.

checkout the script, It runs docker command docker service inspect to find out which service needs to run update command. If in the swarm environment, I believe it will be fine if it runs on manager node.

I'm just too lazy/busy...so I'm fine with running outside of docker for it...But It is a wonderful idea, and PRs are always welcome!

[djmaze]

Just had a quick look this. Yes, the credentials have to be passed somehow. Unfortunately I won't have time to look at this until the week after next. In the meantime, as @caoer said, PR's are welcome ;)

[danieljuhl]

@djmaze I'm more than happy to help - but to be honest.. I have no idea where to start :)

I think for it to really work, the credentials has to be "exposed" from the host. If you'd have to pass the credentials manually to the shepherd service, you would also have to know in advance which services in running and will be running in the swarm.

So ideally, shepherd should (if possible) use the credentials from the host, as Docker already make sure, that all manager nodes is having access to the registry auth, and has the possibility to pass these credentials to the worker nodes.

[danieljuhl]

For your reference, I have already opened an issue in moby/moby#33929

[tbenton]

To work around the problem, we are using secrets to pass the credentials for our registry. Added a docker login inside the shepherd script. Our private repository images now update correctly.

@danieljuhl Thank you for outlining a solution.

[ledermann]

I stumbled upon the same issue and found the following solution:

• Use the fork of @pdeveltere with the WITH_REGISTRY_AUTH env
• Mount /root/.docker/config.json to the service, so the service has access to the credentials

docker service create --name shepherd \
--constraint "node.role==manager" \
--env SLEEP_TIME="1m" \
--env BLACKLIST_SERVICES="shepherd" \
--env WITH_REGISTRY_AUTH="true" \
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock,ro \
--mount type=bind,source=/root/.docker/config.json,target=/root/.docker/config.json,ro \
ledermann/shepherd

This works fine for me (with an image build by myself). Will add a PR about this soon.