Minikube registry instructions don't work on macOS

[mathe-matician]

What Happened?

Following the docker on macOS instructions exactly for setting up a registry with minikube does not work. When trying to run docker push with my newly tagged image, I always encounter Get "http://localhost:5000/v2/": EOF.

Here are the steps I run (after minikube delete):

1. minikube start --cni='cilium' --memory='no-limit' --cpus='no-limit' --nodes=2 --insecure-registry "10.0.0.0/24"
2. minikube addons enable registry
3. The instructions in the docs say "Note: Minikube will generate a port and request you use that port when enabling registry. That instruction is not related to this guide." So I assume I will not use the port that was allocated for me in the subsequent commands and use port 5000. I run docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000"
4. In another terminal I run docker tag <my image>:latest localhost:5000/my_image, docker push localhost:5000/my_image
5. The error that is returned is Get "http://localhost:5000/v2/": EOF
6. The socat container log says 2024/10/16 22:57:07 socat[11] E connect(5, AF=2 192.168.49.2:5000, 16): Connection refused where 192.168.49.2 is the minikube ip.

The only way I've been able to push any image to any registry is to manually start the registry on a node:

1. minikube ssh
2. docker run --restart=always -d -p 5000:5000 --name registry registry:2
3. I can then see within the minikube node via docker ps -a that the new container exists with ports 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp
4. docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000"
5. Then tag and push my image docker push localhost:5000/my_image

Maybe obviously, though, that image will only be available on my control-plane node and not the other one.

Attach the log file

log.txt

Operating System

macOS (Default) arm64

Driver

Docker

[gabor-farkas]

Just had the same issue.

A small difference though, I use minikube with the docker driver, so in this case the registry uses another port, as noted at minikube start:

Registry addon with docker driver uses port 51132 please use that instead of default port 5000

Now, for me at least, when minikube starts up, the port mapping 51132:5000 will be automatically created, so I can use `curl http://localhost:51132/v2/" to get an empty json, as expected.

However, this still doesn't work with docker push, and the reason I found behind that is that it tries to use IPv6.

I used this socat command to get around that:

docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP6-LISTEN:5001,reuseaddr,fork TCP:$(minikube ip):5000"

(Note the TCP6 listen) And then I could use localhost:5001/<imagename> with docker.

[mathe-matician]

@gabor-farkas thanks for your comment. The thing that really fixed it for me was step 1 and 2 below. From the docs, I was just using the subnet that was provided - when switching it to the minikube subnet, and the small modification to the socat step, I got it running via these steps:

1. Run minikube start --help and see the default subnet for the --subnet flag. For me it is 192.168.49.0.
2. Start the minikube cluster using the default subnet as the insecure registry

minikube start -n 2 --insecure-registry "192.168.49.0/24"

3. Run the addon and wait for pods to start

minikube addons enable registry

4. Run the socat container mentioned in the docs. Note to use the host port that is mentioned by the addon's output and not the default port. Here the port was 56866.

docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:56866,reuseaddr,fork TCP:$(minikube ip):5000"

5. Build and tag image

docker tag nginx:latest localhost:56866/nginx:onminikube
docker push localhost:56866/nginx:onminikube

6. Use image in minikube cluster. Ensure it is accessible on all nodes. For the image repo use the output of minikube ip.

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx
  name: nginx
spec:
  nodeName: minikube
  containers:
  - image: 192.168.49.2:5000/nginx:onminikube
    name: nginx
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx-m02
  name: nginx-m02
spec:
  nodeName: minikube-m02
  containers:
  - image: 192.168.49.2:5000/nginx:onminikube
    name: nginx-m02