Access the Containers from Host like on Linux

[KartoffelToby]

This is a kind of a requst

Hello there,

i have a testing development scenario build with docker containers.
on linux machines i can access them via the conatinaer IP (172.17.0.X) and intacting via the exposed Port.

But on Docker for Mac this isn't possible because i dont know the IP from the VM

with the Toolbox (Docker Machine) i can route 172.17.0.x to docker machine ip.. is there any way to do that with Docker for Mac?

I need this because i have multiple Database Containers each with the same Port... (so -p istn't the answer ;))

[dsheets]

Thanks for the feature request. We know this is a useful capability and we have discussed use cases at length.

I'm curious: what prevents you from running multiple database containers and mapping each to a different port and then using a complete sockaddr (address + port rather than address alone) to refer to each?

[KartoffelToby]

It's for a dev environment, so the Developer access databases via domains, it should totaly dynamic so i cant do a static configuration by mapping ports

i resolv the dev domains for the databases (mysql.server1.local to 172.17.0.X)...
On Linux it works like a charm

But on Docker for Mac i have the issue that the 172.17.0.X IPs isn't accessible from Host
because of the VM

[Multiply]

This pretty much relates to #171. Our specific use-case is the same.

We have setup an /etc/resolver file, to use Docker's or Weave's DNS to resolve to local container IPs, and with that we can expect it to 'just work' by using default ports.

Previously, all we needed was to route all traffic into our VM on a specific IP range, but now it's not possible to do so, so we either have to bind to specific ports, for specific services, or we need a clever load balancer, or we need to do manual service discovery.
Either case, the previous method of it just working, and us not having to worry about any of the above, seemed like the better solution to us, at least.

[belooussov]

+1, interested in the fix

[rostchri]

+1, interested in the fix

[aPoCoMiLogin]

Any plans to resolve this issue in near future?

[justincormack]

Hi we are considering how this could be implemented, potentially via a VPN from the Mac to the Linux VM, but it is not being worked on at present. It is probably possible to implement a version by running a container with --net=host running a VPN endpoint too, if someone wants to give it a go.

[aPoCoMiLogin]

@justincormack thanks for fast response. For me and I suppose the rest of people, docker for mac is only for development purpose. So I do not think anyone will bother if this will be easy to use, like docker on linux.

[Kushmaro]

If you're considering VPN I assume there's no way to create network connectivity between Mac and xhyve huh? I wonder why. I think VPN is a bit of an overkill ... really should be a simple developer experience pinging the docker ip's like they would on Linux...

[4shome]

Was the approach mentioned in this thread on the forums considered?

I don't know enough to have an opinion myself, but from reading the first 3 messages, it sounded like there was a fairly simple way to getting a workable solution. Thanks.

[KartoffelToby]

@4shome

Maby i'm wrong but the only Problem is that the xhyve vm inside Docker for Mac hasn't no Network Adapter. Or routable IP like boot2docker (192.168.100.99 like that)

With boot2docker and a route command its possible to Route all the Container ips to the vm Network.

We need this for xhyve.

Whats going on with this tuntap Plugin?

[KartoffelToby]

Here is a abstract scenario that isn't currently working with docker for mac

[aPoCoMiLogin]

Any update on it ?

[bibendi]

🆙 2017

[dimaspivak]

If this were possible, the use case of connecting Docker for Mac containers to overlay networks running across Linux hosts would become possible. This would be so cool for testing (e.g. imagine connecting a locally-developed container with tests to a cloud-based application). Pretty, please? :)

[Mahoney]

As mentioned in the comment above, the Support tap interface for direct container access (incl. multi-host) thread on the Docker forums appears to offer a solution from @michaelhenkel, which I hope he will not mind me copying in here in the hope it moves this issue forward a little:

starting the daemon manually with:

sudo /Applications/Docker.app/Contents/MacOS/com.docker.hyperkit -A -m 4G -c 4 -u -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-tap,tap1 -s 3,virtio-blk,file:///Users/birdman/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2,format=qcow -s 4,virtio-9p,path=/Users/birdman/Library/Containers/com.docker.docker/Data/s40,tag=db -s 5,virtio-rnd -s 6,virtio-9p,path=/Users/birdman/Library/Containers/com.docker.docker/Data/s51,tag=port -s 7,virtio-sock,guest_cid=3,path=/Users/birdman/Library/Containers/com.docker.docker/Data,guest_forwards=2376 -l com1,autopty=/Users/birdman/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty,log=/Users/birdman/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/console-ring -f kexec,/Applications/Docker.app/Contents/Resources/moby/vmlinuz64,/Applications/Docker.app/Contents/Resources/moby/initrd.img,earlyprintk=serial console=ttyS0 com.docker.driverDir="/Users/birdman/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux", com.docker.database="com.docker.driver.amd64-linux" ntp=gateway -F /Users/birdman/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/hypervisor.pid

allows me to assign a routable IP address to moby:

moby:\~# ip addr add 10.1.0.2/24 dev eth0
moby:~# ping 10.1.0.1

PING 10.1.0.1 (10.1.0.1): 56 data bytes
64 bytes from 10.1.0.1: seq=0 ttl=64 time=0.349 ms

and to prove the point:

docker -H tcp://10.1.0.2:2375 network create -d macvlan --subnet 10.1.0.0/24 --ip-range 10.1.0.128/25 --gateway 10.1.0.1 -o parent=eth0 net2

docker -H tcp://10.1.0.2:2375 run -itd --name alp2 --net net2 alpine /bin/sh 93b85398e5356541d8f843c1ce19171cc3c56d217e889c7132dc0c539932c612

docker -H tcp://10.1.0.2:2375 exec -it alp2 ip addr sh

1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: sit0@NONE: mtu 1480 qdisc noop state DOWN qlen 1
link/sit 0.0.0.0 brd 0.0.0.0
3: ip6tnl0@NONE: mtu 1452 qdisc noop state DOWN qlen 1
link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
4: ip6gre0@NONE: mtu 1448 qdisc noop state DOWN qlen 1
link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
14: eth0@ip6gre0: mtu 1500 qdisc noqueue state UNKNOWN
link/ether 02:42:0a:01:00:80 brd ff:ff:ff:ff:ff:ff
inet 10.1.0.128/24 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:aff:fe01:80/64 scope link
valid_lft forever preferred_lft forever

ping 10.1.0.128

PING 10.1.0.128 (10.1.0.128): 56 data bytes
64 bytes from 10.1.0.128: icmp_seq=0 ttl=64 time=0.408 ms
64 bytes from 10.1.0.128: icmp_seq=1 ttl=64 time=0.204 ms

os x can ping the container...

[artemkaint]

I've tried to run docker daemon on sierra and have got

Artems-MacBook-Pro:abcp artemkaint$ sudo /Applications/Docker.app/Contents/MacOS/com.docker.hyperkit -A -m 4G -c 4 -u -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-tap,tap1 -s 3,virtio-blk,file:///Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2,format=qcow -s 4,virtio-9p,path=/Users/artemkaint/Library/Containers/com.docker.docker/Data/s40,tag=db -s 5,virtio-rnd -s 6,virtio-9p,path=/Users/artemkaint/Library/Containers/com.docker.docker/Data/s51,tag=port -s 7,virtio-sock,guest_cid=3,path=/Users/artemkaint/Library/Containers/com.docker.docker/Data,guest_forwards=2376 -l com1,autopty=/Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty,log=/Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/console-ring -f kexec,/Applications/Docker.app/Contents/Resources/moby/vmlinuz64,/Applications/Docker.app/Contents/Resources/moby/initrd.img,earlyprintk=serial console=ttyS0 com.docker.driverDir="/Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux", com.docker.database="com.docker.driver.amd64-linux" ntp=gateway -F /Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/hypervisor.pid
Password:
open of tap device /dev/tap1 failed
mirage_block_open: block_config = file:///Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 and qcow_config = None
com.docker.hyperkit: [INFO] image has 0 free sectors and 244203 used sectors
mirage_block_open: block_config = file:///Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 and qcow_config = None returning 0
mirage_block_stat
virtio-9p: initialising path=/Users/artemkaint/Library/Containers/com.docker.docker/Data/s40,tag=db
virtio-9p: initialising path=/Users/artemkaint/Library/Containers/com.docker.docker/Data/s51,tag=port
vsock init 7:0 = /Users/artemkaint/Library/Containers/com.docker.docker/Data, guest_cid = 00000003
linkname /Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
COM1 connected to /dev/ttys004
COM1 linked to /Users/artemkaint/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
rdmsr to register 0x64e on vcpu 0
rdmsr to register 0x34 on vcpu 0

and script have been paused for a long time

[manfredriem]

It appears it is possible for a standard xhyve install is able to allow access to the xhyve VM from the outside as indicated by the blog post at http://mifo.sk/post/xhyve-for-development

I think this would be the first step to properly supporting --net=host in Docker for Mac

Can someone from the Docker team investigate this?

[jasmas]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well. Would it be possible to get the Developer Preview released for Apple M1 compiled and released on Intel as well? It would be useful so we can test. docker/roadmap#142

[Mahoney]

For anyone else still chasing this - I packaged up @AlmirKadric 's solution as a brew formula:
brew install mahoney/tap/docker-tuntap-osx

Unfortunately there are a couple of manual sudo steps to do as well, listed in the caveats when you do the install.

Works using https://github.com/Mahoney/docker-lifecycle-listener

[Dmitry-N-Medvedev]

For anyone else still chasing this - I packaged up @AlmirKadric 's solution as a brew formula:
brew install mahoney/tap/docker-tuntap-osx

Unfortunately there are a couple of manual sudo steps to do as well, listed in the caveats when you do the install.

Works using https://github.com/Mahoney/docker-lifecycle-listener

FYI:

Warning: Calling depends_on :tuntap is deprecated! There is no replacement.
Please report this issue to the mahoney/tap tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/mahoney/homebrew-tap/Formula/docker-tuntap-osx.rb:10

[Mahoney]

Yes, that's unavoidable. If you want to discuss it further I'd suggest doing so here:
https://github.com/Mahoney/homebrew-tap/issues

[pauldraper]

The Authoritative Guide to Why Docker Choosing Layer 4 Proxy Sucks:

1. It spawns a new process for every port. An FTP server usually needs a sizeable number of ports for data transfer. Making 100 ports available requires 100 processes.
2. It only works for TCP and UDP, excluding common protocols such as ICMP, AH, and ESP.
3. It loses the source IP address, preventing observability and debugging.
4. It ties up a port on the host, making it more difficult to run multiple instances of a service.
5. All these things are regressions compared to VMs.
6. Subjective, but it's really just weird. Docker should not care at all what transport protocols are being used.

[Mahoney]

7. You can't assign a name to a port. You can assign a name to an IP address. Humans are much better at remembering names than numbers. Services have default ports, saving you having to remember any number at all.

[pauldraper]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well.

Does this work?

[blakebarnett]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well.

Does this work?

I verified this works

1. Enable the experimental support in docker-for-mac:
   

2. Find the bridge interface it creates (bridge100 for me), create a route for it, something like:

sudo route -v add -net 172.18.0.1 -netmask 255.255.0.0 192.168.64.2

[blakebarnett]

I had to abandon this for now, the NAT out to the internet is currently broken / flakey. Hoping this improves soon!

[jamshid]

It's not super reliable, especially if you use a VPN on your mac then its --dns option wreaks havoc, but I've used https://github.com/sshuttle/sshuttle (brew install sshuttle) to allow an application running on my mac to have direct TCP access to my containers.

You'll need to know the subnet of your docker network, which can change. E.g. if your containers are in a compose project the network will be mycomposeproject_default.

% docker network inspect mycomposeproject_default | jq -r '.[].IPAM.Config[0].Subnet'
172.21.0.0/16

You'll need to start a container on the same docker network that has sshd running, expose its port 22 as localhost:4222, and set a root password in the container.
Then run sshuttle like:

% sshuttle --dns --no-latency-control --ssh-cmd "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote root@localhost:4222 172.21.0.0/16
[local sudo] Password:  <type your mac password>
root@127.0.0.1's password: <type password for root in your sshd container>
c : Connected to server.
<Ctrl-Z>
zsh: suspended  sshuttle ...
% bg
[1]  + continued  sshuttle ...
% curl -i http://mycontainer   # this works from your mac!
...

It basically proxies all your TCP connections from your mac to that subnet through the container running sshd. It works for me when on VPN except I have to remove --dns and can only access containers by their ip (or I can add /etc/hosts entries for them).

[pauldraper]

I had to abandon this for now, the NAT out to the internet is currently broken / flakey. Hoping this improves soon!

@blakebarnett that might be the same issue as #5680

Which sounds like it's fixed in the upcoming release.

---

P.S. Except for that general issue with the new framework, that route commands works for me.

It does seem that in a docker settings change or upgrade the IP address of the VM can change to another 192.*. FYI.

[blakebarnett]

Yeah I think it is, hopefully in 3.4 everything will be smooth for this!

[Boes-man]

macOS Big Sur v11.4 and Docker Desktop v3.4.0 (65384) still has the issue, as per original request:
But on Docker for Mac this isn't possible because i dont know the IP from the VM
with the Toolbox (Docker Machine) i can route 172.17.0.x to docker machine ip.. is there any way to do that with Docker for Mac?

docker-tuntap-osx is a workaround that works.

[logical-and]

Hey guys, tuntap is a workaround, but when I create a network by
docker network create acme-network
the container becomes unaccesible by previous IP. Did anyone experience such issue and overcome it?

So let's say I have two applications:

app1 with IP 192.168.211.1 and app2 with IP 192.168.213.1. tun/tap does the fix and I can access them by these IPs, but when I add any of applications to new network:

docker network create acme-network
docker network connect acme-network app1
docker network connect acme-network app2

none of the applications are accessible from host like they were before. The reason why I do need to add them to network, because the applications are different containers (with own databases), but I need them work via API.

But due limitation of docker to make them be available to each other I have to create network, and when I do it - containers are not available from host machine.

I tried to add tuntap gateway to docker network, but it didn't help:

docker network inspect acme-network | jq '.[0].IPAM.Config[0]'
sudo route -v add -net 172.19.0.1 -netmask 255.255.0.0 10.0.75.2

Any thoughts, gents?

[Mahoney]

2\. Find the bridge interface it creates (bridge100 for me), create a route for it, someth

Would you mind expanding on this a little? I've checked "Use the new Virtualization framework" on 3.5.2 on an M1 iMac, but ifconfig is only showing one bridge network, bridge0, which is status: inactive and has no IPv4 address.

Output of ifconfig -l: lo0 gif0 stf0 anpi0 anpi1 en0 en4 en5 en2 en3 bridge0 ap1 en1 awdl0 llw0 utun0 utun1 utun2 utun3 utun4 utun5 utun6 utun7

[blakebarnett]

2\. Find the bridge interface it creates (bridge100 for me), create a route for it, someth

Would you mind expanding on this a little? I've checked "Use the new Virtualization framework" on 3.5.2 on an M1 iMac, but ifconfig is only showing one bridge network, bridge0, which is status: inactive and has no IPv4 address.

I upgraded to the Monterey beta on my M1 and haven't tried this out again, I can report back once I do.

[logical-and]

I didn't find a way to do it in this way, but found way how to workaround this - just added app1 network to app2 docker-compose.yml, so basically app2 can connect to network of app1. Since both apps are on docker-compose and have own bridge networks, it works just fine.

Btw, I don't have M1 chip on the system I experienced the issue at

[Mahoney]

Can we keep the conversation focussed on the specific problem the issue is about - communicating with a container by IP address from the host.

This issue is not about communicating with the host from a container.

Nor is it about allowing two different containers to communicate with each other.

This is about being able to do the following on a Mac:

docker run --rm -d --name myservice nginx && \
MYSERVICE_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' myservice ) && \
curl "http://$MYSERVICE_IP"

and seeing the Welcome to nginx! HTML page come back.

If whatever you are suggesting does not make the above snippet work, it's not appropriate for this issue.

[DXist]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well.

Does this work?

I verified this works

1. Enable the experimental support in docker-for-mac:
   
2. Find the bridge interface it creates (bridge100 for me), create a route for it, something like:

sudo route -v add -net 172.18.0.1 -netmask 255.255.0.0 192.168.64.2

It worked on Docker Desktop 3.3.2/3.3.3. Newer versions don't create the bridge interface.

[Mahoney]

Could this be implemented via the new extension API?

https://www.docker.com/blog/docker-extensions-discover-build-integrate-new-tools-into-docker-desktop/

[alexandertsukanov]

There is a brew package that resolves the problem https://github.com/chipmk/docker-mac-net-connect

So you can access the container by IP directly.

Another problem need to be solved is static access to container by its docker's name. (or have it accessible on host machine by port) So I want to achieve behavior identical to--network=host on Linux machine https://docs.docker.com/network/host/.

May be some DNS should work as well, please suggest any solution or workaround.

Please, do not suggest using -p argument in docker run command this is not an option for me.

[logical-and]

@alexandertsukanov , amazing thanks for sharing. Do you guys know any similar solution for Windows? (sorry for offtopic)

[DingYuan0118]

Is there any update for this issue？
I need --network=host feature in mac too.

[gabo-magnet]

Now with the new release 4.14.0 of Docker Desktop on Mac utilizing the Apple Virtualization Framework, it should be possible to add another network adapter featuring the bridged network aka. network_mode = host ?

https://developer.apple.com/documentation/virtualization/vzvirtualmachineconfiguration/3656724-networkdevices

Is this a big challenge? Can somebody point out the hypervisor configuration for the Apple Virtualization Framework here? :)

Also thanks for putting it in the "Considering" space of the docker-roadmap ! 👍

docker/roadmap#238 (comment)

[Mahoney]

There is a brew package that resolves the problem https://github.com/chipmk/docker-mac-net-connect

So you can access the container by IP directly.

Unfortunately docker-mac-net-connect has stopped working with Docker for Mac 4.16.1:
chipmk/docker-mac-net-connect#21