docker-for-win: bug introduced between v5.2.0 and v5.3.0

[eine]

As commented in #92 (comment), with v5.2.0 everything works OK on Windows 10 with docker 18.09.1 and MSYS2. But with v5.3.0 nothing works: no X server is started and no container seems to be created.

The problem is introduced in d773072.

The annoyance to triage this issue is that the x11docker.log file is empty.

# ./x11docker --verbose simexp/octave octave
./x11docker: line 5100: ip: command not found
./x11docker: line 5101: ip: command not found
./x11docker: line 5240: systemctl: command not found

# ./x11docker --debug simexp/octave octave
DEBUGNOTE[739.85]: Command at Line 6686 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6823 - ::main::main
./x11docker: line 6686: PRETTY_NAME: unbound variable
DEBUGNOTE[739.97]:
x11docker version: 5.3.4-beta
docker version:    Docker version 18.09.1, build 4c52b90
Host system:
Command:           ./x11docker '--debug' 'simexp/octave' 'octave'
Parsed options:     --debug -- 'simexp/octave' 'octave'
DEBUGNOTE[740.16]: Command at Line 5082 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6694 - ::check_host::main::main
./x11docker: line 5082: ID: unbound variable
DEBUGNOTE[740.28]: Command at Line 5082 returned with error code 1:
  Hostsystem="$(source /etc/os-release 2>/dev/null; echo $ID)"
  6694 - ::check_host::main::main
./x11docker: line 5100: ip: command not found
./x11docker: line 5101: ip: command not found
DEBUGNOTE[742.43]: ps can not watch root processes.
./x11docker: line 5240: systemctl: command not found
./x11docker: line 5270: XDG_RUNTIME_DIR: unbound variable
DEBUGNOTE[745.72]: Terminating x11docker.
DEBUGNOTE[745.90]: List of stored background processes:

DEBUGNOTE[745.99]: time to say goodbye (finish)
DEBUGNOTE[746.16]: Exitcode 0

# ./x11docker --debug simexp/octave octave
DEBUGNOTE[910.21]: Command at Line 6415 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6549 - ::main::main
./x11docker: line 6415: PRETTY_NAME: unbound variable
DEBUGNOTE[910.40]:
x11docker version: 5.3.0
docker version:    Docker version 18.09.1, build 4c52b90
Host system:
Command:           ./x11docker '--debug' 'simexp/octave' 'octave'
Parsed options:     --debug -- 'simexp/octave' 'octave'
DEBUGNOTE[910.67]: Command at Line 4908 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6423 - ::check_host::main::main
./x11docker: line 4908: ID: unbound variable
DEBUGNOTE[910.85]: Command at Line 4908 returned with error code 1:
  Hostsystem="$(source /etc/os-release 2>/dev/null; echo $ID)"
  6423 - ::check_host::main::main
./x11docker: line 4926: ip: command not found
./x11docker: line 4927: ip: command not found
x11docker note: Could not check whether x11docker is running over ssh.
  Please install pstree.

./x11docker: line 5076: XDG_RUNTIME_DIR: unbound variable
DEBUGNOTE[916.03]: Terminating x11docker.
DEBUGNOTE[916.21]: List of stored background processes:

DEBUGNOTE[916.31]: time to say goodbye (finish)
DEBUGNOTE[916.48]: Exitcode 0

@mviereck, please, let me know if I can do any further tests to help locate the source.

[mviereck]

At the first glance it seems x11docker does not recognize that it runs on Windows and misconfigures itself. I'll look at this.

[mviereck]

I've fixed a bug that already explains the entire fail. I did no test run in Windows VM yet, but maybe it already works now.

[eine]

You are right. It is fixed now. Thanks!

[mviereck]

@1138-4eb
Could you run a test on Windows, please?

$ docker run --init --rm alpine pstree

should give:

init---pstree

Normally x11docker uses init system /usr/bin/docker-init that is not available on MSYS2/Cygwin/WSL.
Maybe it is available in the Docker VM with docker run option --init.
If yes, x11docker could use it and avoid the message:

note "Did not find container init system 'tini'.
  This is a bug in your distributions docker package.
  Normally, docker provides init system tini as '/usr/bin/docker-init'.

  x11docker uses tini for clean process handling and fast container shutdown.
  To provide tini yourself, please download tini-static:
    https://github.com/krallin/tini/releases/download/v0.18.0/tini-static
  Store it in one of:
    $Hostuserhome/.local/share/x11docker/
    /usr/local/share/x11docker/"

[eine]

@1138-4eb
Could you run a test on Windows, please?

Sure! It is correct:

# docker run --init --rm alpine pstree
init---pstree

Normally x11docker uses init system /usr/bin/docker-init that is not available on MSYS2/Cygwin/WSL.
Maybe it is available in the Docker VM with docker run option --init.

This is quite likely. For example, project umarcor/qus provides a script to setup binfmt to use QEMU in order to allow transparent execution of foreign architectures in docker containers. The script does not work on Windows, because it tries to talk to a kernel that does not exist. However, if the script is executed inside a container, it works seamlessly, as the changes are applied to the VM.

If yes, x11docker could use it and avoid the message:

That'd be great.

According to the comment above, I think that, as a rule of thumb, you can run analysis stages of x11docker inside a dummy docker container, instead of doing it on the host only. This will allow to properly detect the environment that will host other containers you start. This feature might be useful in multiple contexts:

• Windows.
• default-runtime: nvidia
• default-runtime: kata-runtime

[mviereck]

@1138-4eb Thank you for testing! The latest commit in master uses docker run option --init and avoids the message about tini.

According to the comment above, I think that, as a rule of thumb, you can run analysis stages of x11docker inside a dummy docker container, instead of doing it on the host only.

x11docker already does a few checks before it starts the desired container in dockerrc.
However, because docker needs root privileges (if not in group docker), x11docker executes docker in dockerrc after everything else is set up already. The final dockerrc scripts contains all commands that need privileges.
It is a design decision not to depend on insecure group docker so x11docker can hardly run container checks in early startup stages.

default-runtime: nvidia
default-runtime: kata-runtime

Are these possible configurations of docker daemon instead of using run option --runtime? You are right that x11docker should check for them if possible.

[eine]

It is a design decision not to depend on insecure group docker so x11docker can hardly run container checks in early startup stages.

Isn't it possible to request the authorization before (and retain it) or either ask for it twice? The user will need to provide it anyway in order to start a container, isn't it?

default-runtime: nvidia
default-runtime: kata-runtime

Are these possible configurations of docker daemon instead of using run option --runtime? You are right that x11docker should check for them if possible.

Yes. Precisely, I found about it because kata-runtime was set up as default in a machine I used. ATM, docker build does not support --runtime (see moby/buildkit#842), so the mechanism to change it is to modify the default and restart the daemon.

See https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file. If /etc/docker/daemon.json does not exist, just create it:

{
"default-runtime": "runc"
}

Custom example for kata-runtime installation (to keep using runc as default):

{
  "default-runtime": "runc",
  "runtimes": {
    "kata-runtime": {
      "path": "/snap/kata-containers/current/usr/bin/kata-runtime",
      "runtimeArgs": [
              "--kata-config /etc/kata/configuration.toml"
      ]
    }
  }
}

This is also explained at https://docs.nvidia.com/dgx/nvidia-container-runtime-upgrade/index.html#using-nv-container-runtime for nvidia.

Nonetheless, you can retrieve this info with docker info, so you don't need to check where it comes from:

# docker info
...
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: kata-runtime runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 09c8266bf2fcf9519a651b04ae54c967b9ab86ec
init version: fec3683
Security Options:
 seccomp
  Profile: default
...

[mviereck]

Let's discuss kata here: #138

Isn't it possible to request the authorization before (and retain it) or either ask for it twice? The user will need to provide it anyway in order to start a container, isn't it?

There is no good way. I would not ask twice, it is already annoying to type the password once.
And I won't store the password anywhere.
x11docker could require to be started as root if the user is not in group docker. But I prefer current solution to interactively ask for the password if needed. It allows to run x11docker as non-root user and non-docker user.