Embed Docker 1.13.1 (but force protocol 1.22) #2277

2opremio · 2017-02-23T15:02:51Z

Fixes #2276 (this embeds a version of the 1.13 client executable, which was split from the docker daemon and thus, it doesn't include sqlite anymore).

For the record, here is proof that docker version 1.10.3 uses protocol 1.22:

$ usr/local/bin/docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      linux/amd64

I still need to test this against older versions of the Docker Engine.

2opremio · 2017-02-23T17:00:44Z

It works on the host, (the docker client returns a 125 status for some commands, maybe due to the mismatching protocol?):

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker version
Client:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64
vagrant@vagrant-ubuntu-wily-64:~/scope$ export PATH=$PWD/docker/docker/:$PATH # use client embedded in scope
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker version # can't talk to the server as is
Client:
 Version:      1.13.1
 API version:  1.24 (downgraded from 1.26)
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 08:47:51 2017
 OS/Arch:      linux/amd64
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.22)
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker/weave ps # it doesn't work with weave either
docker: Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.22).
See 'docker run --help'.                                                                                                                                        
vagrant@vagrant-ubuntu-wily-64:~/scope$ export DOCKER_API_VERSION=1.22 # fake the API protocol                                                                                                                                    
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker version
Client:
 Version:      1.13.1
 API version:  1.22
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 08:47:51 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22 (minimum version )
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64
 Experimental: false
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker/weave ps; echo $? # works but it returns a non-zero status
weave:expose 5e:d7:78:a1:b4:17
125
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker/weave expose; echo $? # same here
10.32.0.1
125
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo docker/weave --local ps weave:expose; echo $? # --local commands (which are the ones we need) do return a 0 status  
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo docker/weave --local expose; echo $?
10.32.0.1
0

But ... it fails when run inside the Scope container, and there is no output:

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22 /usr/bin/weave --local expose; echo $?'                                                      
125
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22 /usr/bin/weave --local ps weave:expose  ; echo $?'
1
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22  /usr/bin/weave --local ps; echo $?'
1

Digging deeper, it seems like docker run is returning a non-zero Status:

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22 ash -x /usr/bin/weave --local ps weave:expose  ; echo $?'
[...]
+ echo 
+ echo -v /var/run/docker.sock:/var/run/docker.sock
+ docker run --rm --privileged --net=host --pid=host -v /var/run/docker.sock:/var/run/docker.sock --entrypoint=/usr/bin/weaveutil weaveworks/weaveexec:1.9.0 container-addrs weave weave:expose
+ CONTAINER_ADDRS=weave:expose weave 5e:d7:78:a1:b4:17 10.32.0.1/12
+ return 1
1

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22  bash -x /usr/bin/weave --local ps; echo $?'
[...]
+++ echo '-v /var/run/docker.sock:/var/run/docker.sock'
++ docker run --rm --privileged --net=host --pid=host -v /var/run/docker.sock:/var/run/docker.sock --entrypoint=/usr/bin/weaveutil weaveworks/weaveexec:1.9.0 container-addrs weave weave:expose 6e820d0fa5d9 1a47c0c90a44 01db383060c7 7836273ab0b8
+ CONTAINER_ADDRS='weave:expose weave 5e:d7:78:a1:b4:17 10.32.0.1/12'
+ return 1

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22 docker run --rm --privileged --net=host --pid=host -v /var/run/docker.sock:/var/run/docker.sock --entrypoint=/usr/bin/weaveutil weaveworks/weaveexec:1.9.0 container-addrs weave weave:expose 6e820d0fa5d9 1a47c0c90a44 01db383060c7 7836273ab0b8; echo $?'
weave:expose weave 5e:d7:78:a1:b4:17 10.32.0.1/12
125

I need to look deeper into this

2opremio · 2017-02-23T17:03:54Z

Actually, it also fails sometimes on the host (but there is output)

vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
127
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0
vagrant@vagrant-ubuntu-wily-64:~/scope$ sudo bash -c "docker/weave --local ps weave:expose; echo $?"
weave:expose 5e:d7:78:a1:b4:17 10.32.0.1/12
0

2opremio · 2017-02-23T17:12:22Z

Apparently 125 means that docker run failed, but it clearly didn't since the command worked:

https://stackoverflow.com/questions/31297616/what-is-the-authoritative-list-of-docker-run-exit-codes

2opremio · 2017-02-23T17:14:12Z

running with debug shows that the hijacker may not be expecting stdout to end?

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22 docker --debug run --rm --privileged --net=host --pid=host -v /var/run/docker.sock:/var/run/docker.sock --entrypoint=/usr/bin/weaveutil weaveworks/weaveexec:1.9.0 container-addrs weave weave:expose 6e820d0fa5d9 1a47c0c90a44 01db383060c7 7836273ab0b8; echo $?'
weave:expose weave 5e:d7:78:a1:b4:17 10.32.0.1/12
DEBU[0000] [hijack] End of stdout                       
125

2opremio · 2017-02-24T13:06:59Z

After rebasing on top of #2278 , this PR seems to work without problems with docker engine 1.10

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker version
Client:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64
vagrant@vagrant-ubuntu-wily-64:~/scope$ ./scope launch                                                                                                                                                     
1b18506d09369d1d7babe66d4f9d200855e6bc844abf0266f77ccbca8d4aa302
Scope probe started
Weave Scope is reachable at the following URL(s):
  * http://10.0.3.1:4040/
  * http://10.32.0.1:4040/
  * http://10.0.2.15:4040/
  * http://172.16.0.3:4040/
vagrant@vagrant-ubuntu-wily-64:~/scope$ docker exec -ti weavescope ash -c 'DOCKER_API_VERSION=1.22  /usr/bin/weave --local ps; echo $?'
weave:expose 26:48:67:7d:c4:f3 10.32.0.1/12
0

The Scope logs also show there are no problems talking to Docker Engine

vagrant@vagrant-ubuntu-wily-64:~/scope$ docker logs weavescope
time="2017-02-24T13:02:17Z" level=info msg="publishing to: 127.0.0.1:4040" 
<probe> INFO: 2017/02/24 13:02:17.397591 command line args: --logtostderr=true --mode=probe --probe.docker=true
<probe> INFO: 2017/02/24 13:02:17.397727 probe starting, version 1e6ced9, ID 75b8a745beced563
<app> INFO: 2017/02/24 13:02:17.404296 app starting, version 1e6ced9, ID 4be536438a7ac351
<app> INFO: 2017/02/24 13:02:17.404357 command line args: --logtostderr=true --mode=app --probe.docker=true
<probe> ERRO: 2017/02/24 13:02:17.429553 Error fetching app details: Get http://127.0.0.1:4040/api: dial tcp 127.0.0.1:4040: getsockopt: connection refused
<app> INFO: 2017/02/24 13:02:17.430963 listening on :4040
<probe> ERRO: 2017/02/24 13:02:17.517001 conntrack stderr:NOTICE: Netlink socket buffer size has been set to 425984 bytes.
<probe> ERRO: 2017/02/24 13:02:17.525430 conntrack stderr:NOTICE: Netlink socket buffer size has been set to 425984 bytes.
<probe> INFO: 2017/02/24 13:02:17.531241 Success collecting weave plugin status
<probe> INFO: 2017/02/24 13:02:17.533064 Success collecting weave status
<app> INFO: 2017/02/24 13:02:17.581604 Success updating weaveDNS
<probe> INFO: 2017/02/24 13:02:17.586810 Success collecting weave proxy status
<probe> INFO: 2017/02/24 13:02:17.606725 Success collecting weave ps
<probe> INFO: 2017/02/24 13:02:18.433178 Control connection to 127.0.0.1 starting
<probe> INFO: 2017/02/24 13:02:20.513497 Publish loop for 127.0.0.1 starting

Kudos to @bboreham

Makefile

@@ -49,6 +49,7 @@ docker/weaveutil:
 $(SCOPE_EXPORT): $(SCOPE_EXE) $(DOCKER_DISTRIB) docker/weave docker/weaveutil $(RUNSVINIT) docker/Dockerfile docker/demo.json docker/run-app docker/run-probe docker/entrypoint.sh
 	cp $(SCOPE_EXE) $(RUNSVINIT) docker/
 	cp $(DOCKER_DISTRIB) docker/docker.tgz
+	cd docker && tar -xvzf docker.tgz


docker/Dockerfile

@@ -4,7 +4,7 @@ LABEL works.weave.role=system
 WORKDIR /home/weave
 RUN apk add --update bash runit conntrack-tools iproute2 util-linux curl && \
 	rm -rf /var/cache/apk/*
-ADD ./docker.tgz /
+ADD ./docker /usr/local/bin/docker


scope

@@ -81,7 +81,7 @@ check_docker_access() {
    fi
 }

-# - The image embeds the weave script & Docker 1.10.3 client
+# - The image embeds the weave script & Docker 1.13.1 client (mimicking a 1.10 client)


2opremio force-pushed the 2276-use-recent-client-as-1.10 branch from 595d2cb to 4dca8b6 Compare February 23, 2017 15:55

2opremio force-pushed the 2276-use-recent-client-as-1.10 branch from 4dca8b6 to 1e6ced9 Compare February 24, 2017 12:55

2opremio changed the title ~~[WIP] Embed Docker 1.13.1 (but force protocol 1.22)~~ Embed Docker 1.13.1 (but force protocol 1.22) Feb 24, 2017

bboreham self-requested a review February 24, 2017 13:11

bboreham requested changes Feb 24, 2017

View reviewed changes

2opremio mentioned this pull request Feb 24, 2017

Add SetEnv() to Cmd mocking interface weaveworks/common#18

Merged

Alfonso Acosta added 4 commits February 24, 2017 13:43

Bump github.com/weaveworks/common

e8cb4fb

Bump google.golang.org/grpc to make github.com/weaveworks/common happy

96e7c6b

Embed Docker 1.13.1 (but force protocol 1.22)

051884d

Review feedback

41c7451

2opremio force-pushed the 2276-use-recent-client-as-1.10 branch from 7ae993d to 41c7451 Compare February 24, 2017 13:50

bboreham approved these changes Feb 24, 2017

View reviewed changes

Review feedback

f892cb1

2opremio merged commit 0ebc0b9 into master Feb 24, 2017

2opremio deleted the 2276-use-recent-client-as-1.10 branch February 24, 2017 14:38

schu mentioned this pull request Mar 7, 2017

integration tests: docker: Error ... iptables: Resource temporarily unavailable. #2308

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Embed Docker 1.13.1 (but force protocol 1.22) #2277

Embed Docker 1.13.1 (but force protocol 1.22) #2277

2opremio commented Feb 23, 2017 •

edited

Loading

2opremio commented Feb 23, 2017 •

edited

Loading

2opremio commented Feb 23, 2017

2opremio commented Feb 23, 2017

2opremio commented Feb 23, 2017

2opremio commented Feb 24, 2017 •

edited

Loading

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

Embed Docker 1.13.1 (but force protocol 1.22) #2277

Embed Docker 1.13.1 (but force protocol 1.22) #2277

Conversation

2opremio commented Feb 23, 2017 • edited Loading

2opremio commented Feb 23, 2017 • edited Loading

2opremio commented Feb 23, 2017

2opremio commented Feb 23, 2017

2opremio commented Feb 23, 2017

2opremio commented Feb 24, 2017 • edited Loading

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

2opremio commented Feb 23, 2017 •

edited

Loading

2opremio commented Feb 23, 2017 •

edited

Loading

2opremio commented Feb 24, 2017 •

edited

Loading