Nomad can't use relative directories in Docker binds in 0.9.2 on Windows

[damoxc]

Nomad version

Nomad v0.9.2 (028326684b9da489e0371247a223ef3ae4755d87)

Operating system and Environment details

Server: Ubuntu 18.04
Client: Windows Server 2019 / Docker 18.09.6

Issue

Relative binds now no longer work under Windows due to the switch to use the Docker volume parser which expects a full path for source and destination.

Reproduction steps

Attempt to use a relative directory in a docker volume spec. The below bit of Go also exhibits the problem:

import (
    "github.com/docker/docker/volume/mounts"
    "errors"
    "fmt"
    "strings"
)

func parseVolumeSpecWindows(volBind string) (hostPath string, containerPath string, mode string, err error) {
	parser := mounts.NewParser("windows")
	m, err := parser.ParseMountRaw(volBind, "")
	if err != nil {
		return "", "", "", err
	}

	src := m.Source
	if src == "" && strings.Contains(volBind, m.Name) {
		src = m.Name
	}

	if src == "" {
		return "", "", "", errors.New("missing host path")
	}

	if m.Destination == "" {
		return "", "", "", errors.New("container path is empty")
	}

	return src, m.Destination, m.Mode, nil
}

func main() {
    volumeDef := "Configuration:C:\\inetpub\\wwwroot\\Configuration\\Local"
    hostPath, containerPath, mode, err := parseVolumeSpecWindows(volumeDef)
    if err != nil {
        fmt.Printf("%+v", err)
    } else {
        fmt.Println("Host Path:", hostPath)
        fmt.Println("Container Path:", containerPath)
        fmt.Println("Mode:", mode)
    }
}

Job file (if appropriate)

I can supply an example if necessary but seems superfluous.

Nomad Client logs (if appropriate)

    2019-06-10T09:03:44.808Z [ERROR] client.driver_mgr.docker: failed to create container configuration: driver=docker image_name=eu.gcr.io/xxx/xxx/xxx:pipeline-xxx image_id=sha256:xxx error="invalid docker volume "Configuration:C:\\inetpub\\wwwroot\\Configuration\\Local": invalid volume specification: 'Configuration:C:\inetpub\wwwroot\Configuration\Local': invalid mount config for type "volume": volume name "configuration" cannot be a reserved word for Windows filenames"
    2019-06-10T09:03:44.808Z [ERROR] client.alloc_runner.task_runner: running driver failed: alloc_id=52c58859-d4de-767b-8179-530b60de60cc task=xxx error="Failed to create container configuration for image "eu.gcr.io/xxx/xxx/xxx:pipeline-xxx" ("sha256:xxx"): invalid docker volume "Configuration:C:\\inetpub\\wwwroot\\Configuration\\Local": invalid volume specification: 'Configuration:C:\inetpub\wwwroot\Configuration\Local': invalid mount config for type "volume": volume name "configuration" cannot be a reserved word for Windows filenames"
    2019-06-10T09:03:44.815Z [INFO ] client.alloc_runner.task_runner: not restarting task: alloc_id=52c58859-d4de-767b-8179-530b60de60cc task=order-xchange reason="Error was unrecoverable"

[notnoop]

@damoxc Can you porvide the job spec, particularly if volume_driver is set.

You are correct parseVolumeSpec doesn't expand the value, but relative paths are expanded when volume_driver is empty which happens in https://github.com/hashicorp/nomad/blob/v0.9.2/drivers/docker/driver.go#L621-L632 with expandPath is defined in https://github.com/hashicorp/nomad/blob/v0.9.2/drivers/docker/utils.go#L208-L216

[damoxc]

@notnoop I can, however the error seems to be coming from parseVolumeSpec (specifically from the Docker library used to perform the parsing on Windows) which occurs before the expansion, which (unless I'm mistaken after reading the code) means the value of volume_driver is irrelevant?

Here's the relevant section from a nomad job inspect though:

"Config": {
    "port_map": [
        {   
            "http": 80.0
        }   
    ],  
    "image": "eu.gcr.io/xxx/xxx/xxx:pipeline-xxx",
    "volumes": [
        "Configuration:C:\\inetpub\\wwwroot\\Configuration\\Local"
    ]   
},

[damoxc]

I've just been debugging and stepping through the Go code above, using the relative path format causes the Docker parser to flag the source as a volume instead of a bind mount.

As it has decided it is a volume not a bind, then it uses https://github.com/moby/moby/blob/master/volume/mounts/windows_parser.go#L261-L274 for validation. The section above for bind mount validation has a call to assert that the path source is an absolute path.

[notnoop]

@damoxc thank you so much! So I misunderstood where the error is coming from - we'll investigate and follow up.

[stale]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!

[stale]

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem 👍

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.