Network Connectivity: How to start the container correctly?

[inkrement]

Hi, I want to connect to the monet-container [172.17.0.7] from another one (lets call it Rcont [172.17.0.2]) . Both are in the same network (bridge) docker0.

If I start monet-container it like this:

docker run -d -P --name $NAME monetdb/monetdb-r-docker

Then I cannot reach the datavase from the host using it's IP or the hosts IP (both commands return something like "connection refused"):

telnet 172.17.0.7 50000 
telnet localhost 50000

And I am also not able to reach it from the other container Rcont (neither on the host' s IP nor the `monet-container's IP).

But when I start it like this:

docker run -d -p 50000:50000 --name $NAME monetdb/monetdb-r-docker

it is not possible to reach the database using the containers IP (telnet 172.17.0.7 50000 from host), but telnet localhost 50000 works on the docker-host (as expected, because I bind the IP).

So I am able to connect from docker-host to the container, but within the container it is a different story. It seems like I am able to reach the database somehow (I get a no route instead of connection refused), but I am not able to connect to it:

root@2ba2030ac77f:/# wget 172.17.0.1:50000
--2017-02-28 16:09:14--  http://172.17.0.1:50000/
Connecting to 172.17.0.1:50000... failed: No route to host.
root@2ba2030ac77f:/# wget 172.17.0.7:50000
--2017-02-28 16:09:19--  http://172.17.0.7:50000/
Connecting to 172.17.0.7:50000... failed: Connection refused.

Although, wget indicates some sort of connection, a minimal example in R (on Rcont) fails:

install.packages("MonetDB.R")
library(MonetDB.R)
library(dplyr)
monetdb_conn <- src_monetdb("demo", host="172.17.0.1", password="MyPassword")

with the error:

Error in socketConnection(host = host, port = port, blocking = TRUE, open = "r+b",  : 
  cannot open the connection
In addition: Warning message:
In socketConnection(host = host, port = port, blocking = TRUE, open = "r+b",  :
  172.17.0.1:50000 cannot be opened

[inkrement]

seems to be realated to #1

[inkrement]

Although, it seems that it is related to #1, I decided to reopen this issue (I have not the right permissions to reopen the other one).

I debugged it a bit and it seems like the daemon is running, but does not respond.

If I connect to the container and start another server (e.g., nohup python -m SimpleHTTPServer &) it answers on incoming connections (e.g., telnet 172.18.0.2 8000) immediatly. However, a connection to the same IP, but monetdb's port (telnet 172.18.0.2 50000) fails.

The daemon itself is running:

[root@82af678b11b5 /]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 112756 12208 ? Ss 15:03 0:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisor/conf.d/su
monetdb 10 0.0 0.0 259676 4104 ? Ssl 15:03 0:00 monetdbd start /var/monetdb5/dbfarm
root 56 0.0 0.0 93868 9988 ? S 15:08 0:00 python -m SimpleHTTPServer
root 60 0.6 0.0 11768 1884 ? Ss 15:16 0:00 /bin/bash
root 75 0.0 0.0 47440 1680 ? R+ 15:16 0:00 ps aux

But it seems to have troubles on startup:

[root@imsm monetdb]# docker run -it --rm --net local monetdb/monetdb-r-docker
2017-03-01 15:18:22,343 CRIT Supervisor running as root (no user in config file)
2017-03-01 15:18:22,346 INFO supervisord started with pid 1
2017-03-01 15:18:23,350 INFO spawned: 'monetdbd' with pid 9
2017-03-01 15:18:23,383 INFO success: monetdbd entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2017-03-01 15:18:23,383 INFO exited: monetdbd (exit status 0; expected)

[inkrement]

@hannesmuehleisen I switched to another docker-image (https://hub.docker.com/r/aquila/monetdb/~/dockerfile/), which works really nice (even without supervisord). The Dockerfile looks really clean, maybe you can use some parts of it.

[hannes]

Hi I am not involved in the Docker image. @kutsurak do you know whats going on here?

[kutsurak]

Hello @inkrement,

The link you provided definitely looks interesting, and we really need to give some attention to docker. @PedroTadim did some work recently on this. I will talk with him and get back to you soon.

[PedroTadim]

Fixing this now. I think the problem is that the CentOS image that we are based from got updated, since the previous compilation of MonetDB Docker. I will put everything back on ASAP.

[PedroTadim]

Found the problem. On MonetDB Jun2016 release, the default listening behavior of the monetdbd ('listenaddr' property) changed to the localhost only. That's why you couldn't connect from the outside of the container. I have just changed the DockerFile in order to the monetdb daemon to listen from any host. You can try by using the latest version of the image and close the issue.

[inkrement]

@PedroTadim have you tested it? I pulled the latest image and started it, but I am not able to reach the database (container is running).

I run it like this:

docker run -d -v /mnt/alpha/data:/data -v /mnt/alpha/container/monetdb/data:/var/monetdb5/dbfarm -p 137.xx.xx.xx:5432:50000 --network local --net-alias monetdb --name monetdb monetdb/monetdb-r-docker

When I start it interactively (without the -d flag) I get the following output:

2017-03-05 18:37:14,073 CRIT Supervisor running as root (no user in config file)
2017-03-05 18:37:14,075 INFO supervisord started with pid 1
2017-03-05 18:37:15,078 INFO spawned: 'monetdbd' with pid 9
2017-03-05 18:37:15,105 INFO success: monetdbd entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2017-03-05 18:37:15,106 INFO exited: monetdbd (exit status 1; not expected)
2017-03-05 18:37:15,106 INFO reaped unknown pid 10

I tried the following commands to reach it (each should work; docker container is running):

• telnet monetdb.local 50000 (from another container within the "local"-network)
• telnet 137.xx.xx.xx 5432 (from a remote machine)
• telnet $(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' monetdb) 50000 (from the docker host)

[PedroTadim]

I got the problem. You are mounting the /var/monetdb5/dbfarm directory, which means you are overriding the merovingian_properties file, setting MonetDB's default listenaddr to the localhost (inside the container only). Maybe we need to rebuild the DockerFile.
Try mounting only the database directory inside the dbfarm as a temporarily solution.

[inkrement]

Strange.. I found this in the documentation

This command mounts the host directory, /src/webapp, into the container at /webapp. If the path /webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content.

... so I thought that this "overlays" means to add files, but not to mask them completely (somehow like the onion file-system works). But I guess you are right... Are you sure that it will work if I overwrite an initialized database? Or do I have to copy all the content before on the host-volume?

Update:

I tried it and changing the path to link just the db folder does work, but without an initial db. Although, the server is running and it is possible to connect, the initial database "db" is not available (cause all the data in this folder is empty). So I tried to start another copy of the image and copied the content of the db folder using docker cp, but this command does not preserve the permissions, so I tried chmod a+rw db. This does not work and it is more or less a dirty "hack". Would you know better/easier ways to persist the data in the container and provide "native-filesystem-performance"? I think this would be interesting for most users.