can't stop weave

[rustjson]

$ docker ps
CONTAINER ID        IMAGE                    COMMAND                CREATED             STATUS              PORTS               NAMES
5592db746a1f        weaveworks/scope:0.6.0   "/home/weave/entrypo   3 days ago          Up 3 days                               weavescope          

~ ⌚ 22:28:17
$ docker stop 55
Error response from daemon: Cannot stop container 55: no such process
FATA[0000] Error: failed to stop one or more containers 

~ ⌚ 22:28:31
$ docker inspect 55 
[{
    "AppArmorProfile": "",
    "Args": [
        "--probe.docker",
        "true"
    ],
    "Config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "--probe.docker",
            "true"
        ],
        "CpuShares": 0,
        "Cpuset": "",
        "Domainname": "",
        "Entrypoint": [
            "/home/weave/entrypoint.sh"
        ],
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "ExposedPorts": {
            "4040/tcp": {}
        },
        "Hostname": "Sunan-ubuntu",
        "Image": "weaveworks/scope:0.6.0",
        "MacAddress": "",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": null,
        "OpenStdin": false,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": "/home/weave"
    },
    "Created": "2015-09-07T16:37:44.614464816Z",
    "Driver": "aufs",
    "ExecDriver": "native-0.2",
    "ExecIDs": null,
    "HostConfig": {
        "Binds": [
            "/var/run/docker.sock:/var/run/docker.sock"
        ],
        "CapAdd": null,
        "CapDrop": null,
        "ContainerIDFile": "",
        "Devices": [],
        "Dns": null,
        "DnsSearch": null,
        "ExtraHosts": null,
        "IpcMode": "",
        "Links": null,
        "LxcConf": [],
        "NetworkMode": "host",
        "PidMode": "host",
        "PortBindings": {},
        "Privileged": true,
        "PublishAllPorts": false,
        "ReadonlyRootfs": false,
        "RestartPolicy": {
            "MaximumRetryCount": 0,
            "Name": ""
        },
        "SecurityOpt": [
            "label:disable"
        ],
        "VolumesFrom": null
    },
    "HostnamePath": "/var/lib/docker/containers/5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c/hostname",
    "HostsPath": "/var/lib/docker/containers/5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c/hosts",
    "Id": "5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c",
    "Image": "5ad5956cf618ce793ccd5ca5309b65634b537ff1aa485c70ded90b03639ec95e",
    "MountLabel": "",
    "Name": "/weavescope",
    "NetworkSettings": {
        "Bridge": "",
        "Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "IPAddress": "",
        "IPPrefixLen": 0,
        "IPv6Gateway": "",
        "LinkLocalIPv6Address": "",
        "LinkLocalIPv6PrefixLen": 0,
        "MacAddress": "",
        "PortMapping": null,
        "Ports": null
    },
    "Path": "/home/weave/entrypoint.sh",
    "ProcessLabel": "",
    "ResolvConfPath": "/var/lib/docker/containers/5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c/resolv.conf",
    "RestartCount": 0,
    "State": {
        "Error": "",
        "ExitCode": 0,
        "FinishedAt": "0001-01-01T00:00:00Z",
        "OOMKilled": false,
        "Paused": false,
        "Pid": 16902,
        "Restarting": false,
        "Running": true,
        "StartedAt": "2015-09-07T16:37:44.793659155Z"
    },
    "Volumes": {
        "/var/run/docker.sock": "/run/docker.sock"
    },
    "VolumesRW": {
        "/var/run/docker.sock": true
    }
}
]

$ ps aux|grep weave
root     22556 28.8  1.2 319184 74596 ?        Sl   22:26   0:53 /home/weave/scope-probe -docker=true
root     22569  1.0  0.1  18524 11248 ?        Sl   22:26   0:01 /home/weave/scope-app

~/docker ⌚ 22:31:07
$ docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef

If I kill that weave process. It will restart. And please notice that inspect return PID are not equals ps PID

[peterbourgon]

The weavescope container runs two processes, scope-{app,probe}, under runit supervision. So, indeed, killing the individual processes will trigger runit to restart them.

The message from Docker indicates that the container (or Docker itself) has got into a bad state somehow. What is the output of ps aux|grep runsv?

[tomwilkie]

Hi Red-wolf-s-husband; could you please send us the output of docker logs weavescope?

I've not seen the use of pids as arguments to docker stop; What happens is you use the container name (ie docker stop weavescope)? Also, could you try stopping with the scope script: scope stop.

[rustjson]

You can have a try to stop container by using docker stop PREFIX-OF(CONTAINER ID)
logs are currently not available, I will post it tonight.

[rustjson]

@tomwilkie

~ ❯ docker logs weavescope
2015/09/07 16:37:44 app starting, version 0.6.0, ID 638fe17a93954b65
2015/09/07 16:37:44 listening on :4040
2015/09/07 16:37:45 probe starting, version 0.6.0, ID Sunan-ubuntu
2015/09/07 16:37:45 publishing to: localhost:4040, scope.weave.local:4040
2015/09/07 16:37:45 docker container: collecting stats for f6b851153c2cf778d8c822badc6563c6822f4c5caf0c568af1adf59e18ad7720
2015/09/07 16:37:45 docker container: collecting stats for 5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c
2015/09/07 16:37:45 docker container: collecting stats for d0ce51feb45e7c938df9361a574e3edffcfc8fdf323b046beff297bccb28abb4
2015/09/07 16:37:45 docker container: collecting stats for 82456990cafca51d7a0e298763a81193fef54354cc57ae901b2fd4dc08a1a23e
2015/09/07 16:37:45 docker container: collecting stats for aff0aa8d822d641a6f49e9a936b79e0a43eaa050e504885b7308904d42ec5593
2015/09/07 16:37:45 docker container: collecting stats for 319b6f8ce97cb38edceea9d1594a3226c0cf99ae593e99e035d21e9f7401a962
2015/09/07 16:37:45 docker container: collecting stats for 5df1e129a1b111078ade80e01357d58ac0917734735b397991420d126868d637
2015/09/07 16:37:45 docker container: collecting stats for 4ff2a78e74a8bf7664d2ff2165bdcb7bb230ff04bb31bdfc9acb377422af878d
2015/09/07 16:37:45 docker container: collecting stats for 0f94e846b869830617f4602fcba66b22fda58aa9324081d4ca61ba2bd336fce1
2015/09/07 16:37:45 docker container: stopped collecting stats for f6b851153c2cf778d8c822badc6563c6822f4c5caf0c568af1adf59e18ad7720
2015/09/08 15:29:01 docker container: collecting stats for e2e04959fc249bc4e9c1ae618ffa5e7b4c727752371f3e5ded364aae16809f43
2015/09/08 15:29:11 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/08 15:29:11 docker container: stopped collecting stats for e2e04959fc249bc4e9c1ae618ffa5e7b4c727752371f3e5ded364aae16809f43
2015/09/08 15:29:34 docker container: collecting stats for e662d5fbc5080cccc03eece97af7e95fb984d27d590c6b57e891c0f74a48175e
2015/09/09 15:18:19 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/09 15:18:19 docker container: stopped collecting stats for e662d5fbc5080cccc03eece97af7e95fb984d27d590c6b57e891c0f74a48175e
2015/09/11 14:20:49 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:20:49 docker container: stopped collecting stats for d0ce51feb45e7c938df9361a574e3edffcfc8fdf323b046beff297bccb28abb4
2015/09/11 14:20:55 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:20:55 docker container: stopped collecting stats for 82456990cafca51d7a0e298763a81193fef54354cc57ae901b2fd4dc08a1a23e
2015/09/11 14:21:00 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:21:00 docker container: stopped collecting stats for aff0aa8d822d641a6f49e9a936b79e0a43eaa050e504885b7308904d42ec5593
2015/09/11 14:21:03 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:21:03 docker container: stopped collecting stats for 319b6f8ce97cb38edceea9d1594a3226c0cf99ae593e99e035d21e9f7401a962
2015/09/11 14:21:07 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:21:07 docker container: stopped collecting stats for 5df1e129a1b111078ade80e01357d58ac0917734735b397991420d126868d637
2015/09/11 14:21:12 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:21:12 docker container: stopped collecting stats for 4ff2a78e74a8bf7664d2ff2165bdcb7bb230ff04bb31bdfc9acb377422af878d
2015/09/11 14:21:14 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:21:14 docker container: stopped collecting stats for 0f94e846b869830617f4602fcba66b22fda58aa9324081d4ca61ba2bd336fce1
2015/09/11 14:26:20 terminated
2015/09/11 14:26:20 docker container: error reading event read unix /var/run/docker.sock: use of closed network connection
2015/09/11 14:26:20 docker container: stopped collecting stats for 5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c
2015/09/11 14:26:21 probe starting, version 0.6.0, ID Sunan-ubuntu
2015/09/11 14:26:21 publishing to: localhost:4040, scope.weave.local:4040
2015/09/11 14:26:21 docker container: collecting stats for 5592db746a1f6d820d939bdd94f48e403533f0659898bef8e9c375fdd43ef11c
2015/09/11 14:26:24 shutting down
2015/09/11 14:26:24 app starting, version 0.6.0, ID 59c328122a37a4bb
2015/09/11 14:26:24 listening on :4040

[tomwilkie]

@peterbourgon is rejigging the shutdown code as part of #424, and we've added a test to ensure this can't happen again.

[rustjson]

大哥，你at错了人！