Can't pull images from an insecure registry in Minikube VM

[dbriones]

This a BUG REPORT

Minikube version v0.9.0:

Environment:

• OS Mac OS X
• VM Driver xhyve
• Docker version 1.12.1
• Install tools:
• Others:

What happened:

We currently host Docker images on a private, insecure registry hosted behind our firewall. (I use a VPN client (openconnect) in order to access the private registry.)

After adding the URL of the private registry in the "Advanced" tab of the Docker.app UI, I was able to use the Docker CLI pull and run images from the private registry without any problem.

I figured that I should be able to do the same thing after starting Minikube and specifying the same registry hostname:port combination that I did in the Docker UI, but using the --insecure-registry flag instead. However, when I tried to run the same image as above, the pod didn't start up, and I saw errors like the following:

Failed to pull image "<image-name>": Error response from daemon: Get https://<registry-hostname>:5000/v1/_ping: EOF

I tried minikube ssh-ing into the VM and noticed that there was no evidence of the --insecure-registry flag being passed to the docker daemon command. I expected to see it included in EXTRA_ARGS in /var/lib/boot2docker/profile. I tried adding the flag to that file and restarting the Docker daemon -- and was then able to pull and run images from the private registry as expected. Of course, when I stopped and restarted the Minikube VM, my changes were lost and I was no longer able to pull from the private registry.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else do we need to know:

[dlorenc]

ref #523

Hey, can you confirm how you used the --insecure-registry flag? I think we currently have a bug where you have to pass that flag after running "minikube stop".

[dbriones]

Sure, I passed it as a flag to minikube start:

minikube start --vm-driver="xhyve" --insecure-registry="<registry-hostname>:5000"

[aecolley]

I just tried this with minikube v0.10.0 and --insecure-registry='docker-registry.example.com:443' was correctly copied into /var/lib/boot2docker/profile in the new xhyve VM. However, as @dlorenc hinted, the --insecure-registry flag is ignored if the machine already existed (even if it is stopped). You must first minikube delete if you want new flags to be respected (see #523 ).

[dlorenc]

Thans for confirming @aecolley. We can close this out once we get #523 in.

[Starefossen]

Does anyone know the status of #523 or should we add a note about this (limitation) to https://github.com/kubernetes/minikube/blob/master/docs/minikube_start.md.

[dbriones]

Thanks, @aecolley -- I deleted the VM and started it again with the flags above and it works as expected.

Looking at #523, I'm wondering whether the semantics of the subcommands are causing confusion. I think I was expecting that start-ing a VM would be stateless, given that the subcommand accepts the --vm-driver and --insecure-registry flags. Since it is actually stateful (and seemingly, immutable), maybe it make sense for those flags to be moved from start to new subcommand that implies statefulness; perhaps create?

[surajssd]

same with kvm as well, I tried deleting the VM and pulling iso again but it didn't help. No entry in /var/lib/boot2docker/profile with flag --insecure-registry

[cesargomezvela]

Finally I found the solution without deleting the VM here: http://stackoverflow.com/a/39698096/5408030

Also, I show you the content of my profile archive
EXTRA_ARGS='
--label provider=xhyve
--insecure-registry=192.168.64.2:30873
'
CACERT=/var/lib/boot2docker/ca.pem
DOCKER_HOST='-H tcp://0.0.0.0:2376'
DOCKER_STORAGE=aufs
DOCKER_TLS=auto
SERVERKEY=/var/lib/boot2docker/server-key.pem
SERVERCERT=/var/lib/boot2docker/server.pem

I copy the post below

For an http registry this steps works for me:

1. minikube ssh

2. edit /var/lib/boot2docker/profile and add to $EXTRA_ARGS --insecure-registry yourdomain.com:5000

3. restart the docker daemon sudo /etc/init.d/docker restart

[aisrael]

Just wanted to add a note that as of minikube 0.19.0 (and maybe earlier, but I only got this now) minikube uses systemd, hence:

minikube ssh
sudo systemctl restart docker

[gabencui]

@aecolley thank you very match!
"the --insecure-registry flag is ignored if the machine already existed (even if it is stopped). You must first minikube delete if you want new flags to be respected"
is correct.

[kunalg]

@gabencui @aecolley You don't need to delete and re-create the VM just to pass --insecure-registry.

There is a JSON file created after the minikube VM is created (I used virtualbox on linux) -
$HOME/.minikube/machines/minikube/config.json - that seems to contain a lot of config for the VM.

Here is mine:
https://gist.github.com/kunalg/015aacb58d18bd110844922da7329c22

In this, you can see some interesting config under 'HostOptions' -> 'EngineOptions' - including the 'InsecureRegistry'.

You can edit this JSON array while your minikube VM is stopped. And, once restarted, it seems to take effect - without deleting the VM 😌 😁

This is the output from my current minikube vm (minikube ssh):

$ systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: active (running) since Fri 2017-06-16 17:18:44 UTC; 2 days ago
     Docs: https://docs.docker.com
           https://docs.docker.com
 Main PID: 3061 (docker)
    Tasks: 184
   Memory: 615.6M
      CPU: 31min 50.245s
   CGroup: /system.slice/docker.service
           ├─2809 docker-containerd-shim 6a777123b7d6e104e9648384ac22010aa3f1057b45e69e13938660e769d3a26f /var/run/docker/libcontainerd/6a777123b7d6e104e9648384ac22010aa3f1057b45e69e13938660e769d3a26f docker-runc
           ├─3061 /usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 192.168.99.1:5000
           ├─3088 docker-containerd -l /var/run/docker/libcontainerd/docker-containerd.sock --runtime docker-runc
           ├─3454 docker-containerd-shim 7847cef9dd3121421ed08ef36339f65630f46d9edb4ed3608f781aa9c886ad78 /var/run/docker/libcontainerd/7847cef9dd3121421ed08ef36339f65630f46d9edb4ed3608f781aa9c886ad78 docker-runc
           ├─3506 docker-containerd-shim fbd5cbe70c1baa64801e57a1a02aef4bc7f394b0023f7e5239e64d28bf88410d /var/run/docker/libcontainerd/fbd5cbe70c1baa64801e57a1a02aef4bc7f394b0023f7e5239e64d28bf88410d docker-runc
           ├─3586 docker-containerd-shim 7a8c730b42e59f49ad8f3e48955e7802db0c4399714e241d89a762654f31f236 /var/run/docker/libcontainerd/7a8c730b42e59f49ad8f3e48955e7802db0c4399714e241d89a762654f31f236 docker-runc
           ├─3762 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 18080 -container-ip 172.17.0.4 -container-port 18080
           ├─3795 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.17.0.4 -container-port 443
           ├─3809 docker-containerd-shim 17ba3c733bb58e2caf51af8d062c961864d815620da0d92a23ece4f9d811332a /var/run/docker/libcontainerd/17ba3c733bb58e2caf51af8d062c961864d815620da0d92a23ece4f9d811332a docker-runc
           ├─3814 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.17.0.4 -container-port 80
           ├─3857 docker-containerd-shim 6af25cb29ccfb82c902b201a45fa2b7d795a9a86182ca9443da43250e7528018 /var/run/docker/libcontainerd/6af25cb29ccfb82c902b201a45fa2b7d795a9a86182ca9443da43250e7528018 docker-runc
           ├─3859 docker-containerd-shim 5b14db4a4bf5d57ed04aa34cfc7baf0fed521fedb71fae60ddc717ccb6a1b35d /var/run/docker/libcontainerd/5b14db4a4bf5d57ed04aa34cfc7baf0fed521fedb71fae60ddc717ccb6a1b35d docker-runc
           ├─4000 docker-containerd-shim 249aa4921f43af0f2e7ee4bd9f8df87b7b975f3b15828221753eb5373ff6ef6c /var/run/docker/libcontainerd/249aa4921f43af0f2e7ee4bd9f8df87b7b975f3b15828221753eb5373ff6ef6c docker-runc
           ├─4065 docker-containerd-shim 0bda4419c09c5c1a6d0cd29bf5b6d25f0c13636888eba6067b3afda79f89dd64 /var/run/docker/libcontainerd/0bda4419c09c5c1a6d0cd29bf5b6d25f0c13636888eba6067b3afda79f89dd64 docker-runc
           ├─4116 docker-containerd-shim 2b8ebcdb17451604f237eaa9c92d580dc613ef19d4026d15313a515291b6292f /var/run/docker/libcontainerd/2b8ebcdb17451604f237eaa9c92d580dc613ef19d4026d15313a515291b6292f docker-runc
           ├─4198 docker-containerd-shim b3344fc0a219142a84792fc9a1f0c03aacf259eb7d582f49315cdff074378427 /var/run/docker/libcontainerd/b3344fc0a219142a84792fc9a1f0c03aacf259eb7d582f49315cdff074378427 docker-runc
           ├─4470 docker-containerd-shim 974f5751887d394e41b2817a613805690c0928155777bb6c80949bc813420e19 /var/run/docker/libcontainerd/974f5751887d394e41b2817a613805690c0928155777bb6c80949bc813420e19 docker-runc
           ├─4884 docker-containerd-shim bf40c06301a4916a7cbea5083d06a809bfc6876012cb2d0ad0564092ee312b07 /var/run/docker/libcontainerd/bf40c06301a4916a7cbea5083d06a809bfc6876012cb2d0ad0564092ee312b07 docker-runc
           └─4936 docker-containerd-shim 0c162ec316408c4f1766322d12dc101e642b3db4d813d14735b62d74fc4bd46d /var/run/docker/libcontainerd/0c162ec316408c4f1766322d12dc101e642b3db4d813d14735b62d74fc4bd46d docker-runc

If you look at process id 3061, it carries the --insecure-registry flag.

Now, following output is after stopping the VM, editing the config.json file to remove the insecure registry entry and restarting the vm:

$ systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: active (running) since Sun 2017-06-18 18:38:26 UTC; 59s ago
     Docs: https://docs.docker.com
           https://docs.docker.com
 Main PID: 3049 (docker)
    Tasks: 175
   Memory: 60.2M
      CPU: 2.953s
   CGroup: /system.slice/docker.service
           ├─3049 /usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox
           ├─3079 docker-containerd -l /var/run/docker/libcontainerd/docker-containerd.sock --runtime docker-runc
           ├─3470 docker-containerd-shim 86c2d37aedcffd56f6d02d807f93cb685ae1fcbb74eedd1cc52925751d59d455 /var/run/docker/libcontainerd/86c2d37aedcffd56f6d02d807f93cb685ae1fcbb74eedd1cc52925751d59d455 docker-runc
           ├─3525 docker-containerd-shim 2a32423f150c4e5efbc3f4a981be95532b26861be528947dc32cc97bff2dbf6f /var/run/docker/libcontainerd/2a32423f150c4e5efbc3f4a981be95532b26861be528947dc32cc97bff2dbf6f docker-runc
           ├─3609 docker-containerd-shim a866be39ee460111f8b5cbfdae48e5751a7af3cde5f3d26357ab6867c54dcb5a /var/run/docker/libcontainerd/a866be39ee460111f8b5cbfdae48e5751a7af3cde5f3d26357ab6867c54dcb5a docker-runc
           ├─3673 docker-containerd-shim 2298dcc0318f3a495669ebadd60240794b814c9693b2310478b56f62f4a2670f /var/run/docker/libcontainerd/2298dcc0318f3a495669ebadd60240794b814c9693b2310478b56f62f4a2670f docker-runc
           ├─3750 docker-containerd-shim 06d507f573f4dd3c38d912e26114900f71873db40727292d49eb8d8e7174c02e /var/run/docker/libcontainerd/06d507f573f4dd3c38d912e26114900f71873db40727292d49eb8d8e7174c02e docker-runc
           ├─3754 docker-containerd-shim b1efc9febec0eb283e5e527a87ce284150bf515897670e06b92e0dbb662e6d25 /var/run/docker/libcontainerd/b1efc9febec0eb283e5e527a87ce284150bf515897670e06b92e0dbb662e6d25 docker-runc
           ├─3871 docker-containerd-shim 4a8b286e3cc6d96316bfc1f364766a320e7dffdc4dcef74da09c68d79bbf5061 /var/run/docker/libcontainerd/4a8b286e3cc6d96316bfc1f364766a320e7dffdc4dcef74da09c68d79bbf5061 docker-runc
           ├─3926 docker-containerd-shim ac969b1a2c9ae674a83ee172b10dc60f4339b30409ff6a1b22e1b529be6af37f /var/run/docker/libcontainerd/ac969b1a2c9ae674a83ee172b10dc60f4339b30409ff6a1b22e1b529be6af37f docker-runc
           ├─3962 docker-containerd-shim d89dc294e265020b696b008aa5bbdf3656aa5a61d7f4ef68b3482778c91d7e1a /var/run/docker/libcontainerd/d89dc294e265020b696b008aa5bbdf3656aa5a61d7f4ef68b3482778c91d7e1a docker-runc
           ├─4011 docker-containerd-shim 515be31ac9b6342c597d618498731a6772fe0ffc7063a52b24186f10f0a46f2a /var/run/docker/libcontainerd/515be31ac9b6342c597d618498731a6772fe0ffc7063a52b24186f10f0a46f2a docker-runc
           ├─4060 docker-containerd-shim 12ca7b9024a5e108bb65e30ab46e1531fb59c68c0d022d47f36cb46e361c7d53 /var/run/docker/libcontainerd/12ca7b9024a5e108bb65e30ab46e1531fb59c68c0d022d47f36cb46e361c7d53 docker-runc
           ├─4156 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 18080 -container-ip 172.17.0.6 -container-port 18080
           ├─4165 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.17.0.6 -container-port 443
           ├─4173 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.17.0.6 -container-port 80
           ├─4187 docker-containerd-shim 9c65c289e1f90f5d022207edf8f86412661937fb20576652fdc7033ddffdf759 /var/run/docker/libcontainerd/9c65c289e1f90f5d022207edf8f86412661937fb20576652fdc7033ddffdf759 docker-runc
           ├─4236 docker-containerd-shim 9ffc6e01199d967257f9f8ddaf615136ae0aaa693f3a6d3073680898f3f0083a /var/run/docker/libcontainerd/9ffc6e01199d967257f9f8ddaf615136ae0aaa693f3a6d3073680898f3f0083a docker-runc
           └─4304 docker-containerd-shim 993bf7a64a3a731e436cb82afcc4307227142db582cc0543fbdc47fcfe515a68 /var/run/docker/libcontainerd/993bf7a64a3a731e436cb82afcc4307227142db582cc0543fbdc47fcfe515a68 docker-runc

Now, if you see process id 3049 in the above output, there is no --insecure-registry passed.

This is after adding the entry again:

$ systemctl status docker                                 
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: active (running) since Sun 2017-06-18 18:42:38 UTC; 1min 26s ago
     Docs: https://docs.docker.com
           https://docs.docker.com
 Main PID: 2922 (docker)
    Tasks: 182
   Memory: 63.4M
      CPU: 3.138s
   CGroup: /system.slice/docker.service
           ├─2922 /usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 192.168.99.1:5000
           ├─2950 docker-containerd -l /var/run/docker/libcontainerd/docker-containerd.sock --runtime docker-runc
           ├─3365 docker-containerd-shim 62b5787414c2f247c71562716732f235ef72d5cb3128d23650b2c5995bb19834 /var/run/docker/libcontainerd/62b5787414c2f247c71562716732f235ef72d5cb3128d23650b2c5995bb19834 docker-runc
           ├─3417 docker-containerd-shim a2466c5d3e47cd6fd6961b0a54987527e2f1b61dc802074922a50e56b8fbd025 /var/run/docker/libcontainerd/a2466c5d3e47cd6fd6961b0a54987527e2f1b61dc802074922a50e56b8fbd025 docker-runc
           ├─3519 docker-containerd-shim 9983df8d102b0102c21b53f5dc1ad77447777bdaae908712103632acb9ff8758 /var/run/docker/libcontainerd/9983df8d102b0102c21b53f5dc1ad77447777bdaae908712103632acb9ff8758 docker-runc
           ├─3566 docker-containerd-shim 59d3a76bf96fe06b7532315843ce362c0453f14f2f997d0fcf1786d564fc352a /var/run/docker/libcontainerd/59d3a76bf96fe06b7532315843ce362c0453f14f2f997d0fcf1786d564fc352a docker-runc
           ├─3640 docker-containerd-shim 2455788a4bfaee69170aa4b7f76f496c05ed1c53fffaf222f4b45c5b5e6ebf1d /var/run/docker/libcontainerd/2455788a4bfaee69170aa4b7f76f496c05ed1c53fffaf222f4b45c5b5e6ebf1d docker-runc
           ├─3671 docker-containerd-shim a9553273627afd202b5eb226a57c390d23f7b91362dd595e7f144a1068b1b509 /var/run/docker/libcontainerd/a9553273627afd202b5eb226a57c390d23f7b91362dd595e7f144a1068b1b509 docker-runc
           ├─3752 docker-containerd-shim c7163f71ed87525a94de87ac5aab151860684bbecc51fa7cf7bbd30ed3c33552 /var/run/docker/libcontainerd/c7163f71ed87525a94de87ac5aab151860684bbecc51fa7cf7bbd30ed3c33552 docker-runc
           ├─3819 docker-containerd-shim 4c2b0c19d2929737cfc53367dfd0e20a0945136098e0c47823b4c0c2dd75ef3f /var/run/docker/libcontainerd/4c2b0c19d2929737cfc53367dfd0e20a0945136098e0c47823b4c0c2dd75ef3f docker-runc
           ├─3906 docker-containerd-shim fd68f54489ec320dc66980677f9cd37cf759b4a28018a8658b5df83ff6990d3e /var/run/docker/libcontainerd/fd68f54489ec320dc66980677f9cd37cf759b4a28018a8658b5df83ff6990d3e docker-runc
           ├─3966 docker-containerd-shim 0d916d0bc26bf71582743fff9210bb6d1b2b16f33595385acf906b8a28315379 /var/run/docker/libcontainerd/0d916d0bc26bf71582743fff9210bb6d1b2b16f33595385acf906b8a28315379 docker-runc
           ├─4025 docker-containerd-shim 491fc9f75d10bf5aa0103ea587318c24aee0ec453806da01ecf21231bba77439 /var/run/docker/libcontainerd/491fc9f75d10bf5aa0103ea587318c24aee0ec453806da01ecf21231bba77439 docker-runc
           ├─4064 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 18080 -container-ip 172.17.0.6 -container-port 18080
           ├─4094 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.17.0.6 -container-port 443
           ├─4106 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.17.0.6 -container-port 80
           ├─4112 docker-containerd-shim 4f2af8bb6d39dbcce9b45e8c82484421bab09128fde91736110459314c354db6 /var/run/docker/libcontainerd/4f2af8bb6d39dbcce9b45e8c82484421bab09128fde91736110459314c354db6 docker-runc
           ├─4187 docker-containerd-shim 3cefdfc9f9c6aa1c0d8412fc06d1fa6d3034b51c7f6f355ecd4f622d886c0174 /var/run/docker/libcontainerd/3cefdfc9f9c6aa1c0d8412fc06d1fa6d3034b51c7f6f355ecd4f622d886c0174 docker-runc
           └─4284 docker-containerd-shim e9494013e94f6c30d0109203a1a4027c7613bfe99733b27193b0c63ef6246c1c /var/run/docker/libcontainerd/e9494013e94f6c30d0109203a1a4027c7613bfe99733b27193b0c63ef6246c1c docker-runc

And voilà, --insecure-registry flag back again in process id 2922. 😉

[lingxiankong]

Hi, @kunalg, did you try to pull images after configuring '--insecure-registry', seems docker was still trying to pull image from docker.io, my output:
https://hastebin.com/kiyequbeni.js

[kunalg]

@lingxiankong Yes, I was able to pull images in kubernetes from my registry by explicitly referencing it in the --image=.... parameter as:

$ kubectl run hello-node --image=192.168.99.1:5000/hello-node:v3 --port=9099

where my insecure registry runs at 192.168.99.1:5000 as outlined earlier.
hello-node is the image name and :v3 specifies the image tag.

I believe, the same should work in a yaml definition file for pod/service where you need to specify an image.

[lingxiankong]

@kunalg thanks. To be clear, I can also pull the image by specifying the registry url explicitly. But how can we pull the image only using 'hello-node:v3'? We have already configured the registry path.

[kunalg]

@lingxiankong I believe, that's not going to be possible.
AFAIK, when no host is specified, docker defaults to docker hub registry (registry-X.docker.io).
Even Google container registry (GCR) requires the hostname/project-id to be specified:
https://cloud.google.com/container-registry/docs/quickstart#testing_your_image

Even in minikube, the kubelet images seem to be pulled from host gcr.io (output is from inside minikube vm):

$ docker images
REPOSITORY                                             TAG                 IMAGE ID            CREATED             SIZE
192.168.99.1:5000/hello-node                           v4                  f137e316ffe7        5 days ago          649.6 MB
gcr.io/google_containers/kubernetes-dashboard-amd64    v1.6.1              71dfe833ce74        5 weeks ago         134.4 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64         1.14.2              7c4034e4ffa4        6 weeks ago         44.5 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64        1.14.2              ca8759c215c9        6 weeks ago         52.36 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64   1.14.2              e5c335701995        6 weeks ago         44.84 MB
gcr.io/google_containers/nginx-ingress-controller      0.9.0-beta.4        a811f37ddd0e        8 weeks ago         120.4 MB
gcr.io/google-containers/kube-addon-manager            v6.4-beta.1         85809f318123        3 months ago        127.2 MB
192.168.99.1:5000/hello-node                           v3                  839da3ec37c7        13 months ago       649.6 MB
gcr.io/google_containers/pause-amd64                   3.0                 99e59f495ffa        13 months ago       746.9 kB
gcr.io/google_containers/defaultbackend                1.0                 137a07dfd084        20 months ago       7.51 MB

Similary, AWS ECR requires the hostname:
http://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-pull-ecr-image.html

[malcolmm83]

This didn't work for me. When I restarted minikube it just overwrote the changes. I'm able to do this in my dockertoolbox VM by just editiing /var/lib/boot2docker/profile Not sure why it's so hard here. When you say stop the minikube VM, there's lots of ways to do that. Which did you have in mind?

[vhosakot]

I saw the same error too when I ran docker build -t hello-node:v1 . in the tutorial https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/#create-a-docker-container-image on Mac OS X Sierra (10.12.6), Docker version 17.12.0-ce, minikube version v0.25.0, xhyve version 0.2.0 and kubernetes version v1.9.0:

$ docker build -t hello-node:v1 .
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM node:6.9.2
Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.64.1:53: read udp 192.168.64.3:37095->192.168.64.1:53: read: connection refused

Switching to virtualbox fixed this issue for me.

I stopped minikube, deleted it, started it without --vm-driver=xhyve (minikube uses virtualbox driver by default), and then docker build -t hello-node:v1 . worked fine without errors:

minikube stop
eval $(minikube docker-env -u)
minikube delete

minikube start (without --vm-driver=xhyve)

eval $(minikube docker-env)
docker build -t hello-node:v1 .

$ docker images | grep hello
hello-node                                    v1                  9cc51aa82a30        39 seconds ago      655MB

I do see the kube-dns pod running:

$ kubectl get pod --all-namespaces
NAMESPACE     NAME                                    READY     STATUS    RESTARTS   AGE
kube-system   kube-addon-manager-minikube             1/1       Running   0          1m
kube-system   kube-dns-54cccfbdf8-vmvjm               3/3       Running   0          1m
kube-system   kubernetes-dashboard-77d8b98585-5mbcf   1/1       Running   0          1m
kube-system   storage-provisioner                     1/1       Running   0          1m

This issue does look like an xhyve issue not seen with virtualbox. My virtualbox version is 5.2.6r120293:

$ VBoxManage --version
5.2.6r120293

[ffroliva]

Is it possible to set multiple insecure repositores to --insecure-registry in a comma separated approache?

For example:
minikube start --insecure-registry="nexus:8888,nexus:8889 "

[BurlyLuo]

root@vm16040:~# cat /etc/docker/daemon.json 
{
"exec-opts": ["native.cgroupdriver=cgroupfs"],
"log-driver": "json-file",
"log-opts": {
        "max-size": "100m"
},
"storage-driver": "overlay2"
}
root@vm16040:~# minikube start --insecure-registry="192.168.2.100:5000" --image-mirror-country='cn' --kubernetes-version=v1.20.15 --vm-driver=none --forcec

The issue is still here when the driver=none.
why??? it's hard to do? i think no. just add the "insecure-registries":["192.168.2.100:5000"],
into:/etc/docker/daemon.json

b4 start the k8s pod(docker container), we can add this line into the daemon.json. and then restart the docker service. and then start the k8s system pods.
so terible experience.

[nicolassnight]

I'm running Minikube with driver=none on Debian 12 VM
Works for me: when minikube is started, adding insecure registry as mentioned in https://stackoverflow.com/questions/42211380/add-insecure-registry-to-docker just on my VM and then systemctl restart docker