Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[18.09 backport] bump Golang 1.11.11, and some makefile improvements #1709

Merged
merged 15 commits into from
Jul 23, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Mar 4, 2019

Updates the 18.09 branch to Go 1.11.x, because Go 1.10 reached EOL

docker/cli:

git checkout -b 18.09_backport_bump_golang_1.11 upstream/18.09

# https://github.com/docker/cli/pull/1426 Bump Go to 1.11.1
git cherry-pick -s -S -x 51848bf3bb3c6cad43ead02ef0660e47e741d2d5
git cherry-pick -s -S -x ee461303f9e74937a0a598650249f4331b1dd498
git cherry-pick -s -S -x 941273918665ddbf2ba062c0889e443882a90825
git cherry-pick -s -S -x 906c2d161a6ad4ccd3c6d16134385d619d1f7cc9

# https://github.com/docker/cli/pull/1537 do not patch Dockerfiles in CI
git cherry-pick -s -S -x 166856ab1b1681de107b5e064ea89ea4b27154fd

# https://github.com/docker/cli/pull/1558 Add a `fmt` build target and make use of a go build cache.
git cherry-pick -s -S -x 7c8ee78eaf69106beac78d9bf69e5034cfff8f9d
git cherry-pick -s -S -x d5de8358f0192551970965d8fabb9a967f0f51f1


# https://github.com/docker/cli/pull/1545 Bump Go to 1.11.2
git cherry-pick -s -S -x 58f0bfcf51536312d896738ec6c6d1ec73bc2caf


# https://github.com/docker/cli/pull/1585 Bump Golang 1.11.4 (includes fix for CVE-2018-16875)
git cherry-pick -s -S -x deaf6e13ab067e6794d20ec980b4ae216b65d07c

# https://github.com/docker/cli/pull/1636 Bump Golang 1.11.5 (CVE-2019-6486)
git cherry-pick -s -S -x 0e9d1d3b071f09f96d800b55aa3c2fb715dc179b

# https://github.com/docker/cli/pull/1742 Bump Golang 1.11.6
git cherry-pick -s -S -x 1500105975b29beb9393c79fe5652e901cbd5966

@codecov-io
Copy link

codecov-io commented Mar 4, 2019

Codecov Report

Merging #1709 into 18.09 will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##            18.09    #1709   +/-   ##
=======================================
  Coverage   54.23%   54.23%           
=======================================
  Files         290      290           
  Lines       19428    19428           
=======================================
  Hits        10537    10537           
  Misses       8212     8212           
  Partials      679      679

@thaJeztah thaJeztah changed the title [18.09 backport] bump Golang 1.11.5, and some makefile improvements [18.09 backport] bump Golang 1.11.6, and some makefile improvements Mar 17, 2019
@thaJeztah thaJeztah modified the milestones: 18.09.4, 18.09.5 Mar 28, 2019
@thaJeztah
Copy link
Member Author

16:41:55 Creating mailbox file: No such file or directory
16:41:55 Password: chsh: PAM: Authentication token manipulation error

need to rebase to get #1729

@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch from 44dfcf6 to 0a6422b Compare March 28, 2019 17:18
@thaJeztah thaJeztah modified the milestones: 18.09.5, 18.09.6 Mar 28, 2019
@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch from 0a6422b to 2e6a04f Compare March 28, 2019 17:48
Copy link
Collaborator

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🐯

@thaJeztah
Copy link
Member Author

fwiw; discussed this with @andrewhsu, and given that this is not critical, and changes in other repos will have to be made as well, we'll skip this for the 18.09.5 release, but will include it in 18.09.6, so don't merge yet

@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch 2 times, most recently from 74b32fa to 7fac617 Compare April 7, 2019 22:55
@thaJeztah thaJeztah changed the title [18.09 backport] bump Golang 1.11.6, and some makefile improvements [18.09 backport] bump Golang 1.11.7, and some makefile improvements Apr 7, 2019
@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch from 7fac617 to b344d48 Compare April 14, 2019 00:25
@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch from b344d48 to c6060b4 Compare May 20, 2019 17:32
@thaJeztah thaJeztah changed the title [18.09 backport] bump Golang 1.11.7, and some makefile improvements [18.09 backport] bump Golang 1.11.10, and some makefile improvements May 20, 2019
@thaJeztah thaJeztah modified the milestones: 18.09.6, 18.09.7 Jun 14, 2019
kolyshkin and others added 5 commits June 14, 2019 12:52
Fix this warning from go-1.11

> cli/registry/client/fetcher.go:234: Debugf format %s has arg
> repoEndpoint of wrong type client.repositoryEndpoint

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 51848bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In case go build will see a need to call C++ (rather than C)
compiler, CXX env var need to be properly set (to osxcross wrapper).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ee46130)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Release notes: https://golang.org/doc/devel/release.html#go1.11

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9412739)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gofmt/goimports changed some heuristics in 1.11 and the code is now
formatted slightly differently.

No functional change, just whitespace.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 906c2d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building the Dockerfiles for development, those images are mainly used to
create a reproducible build-environment. The source code is bind-mounted into
the image at runtime; there is no need to create an image with the actual
source code, and copying the source code into the image would lead to a new
image being created for each code-change (possibly leading up to many "dangling"
images for previous code-changes).

However, when building (and using) the development images in CI, bind-mounting
is not an option, because the daemon is running remotely.

To make this work, the circle-ci script patched the Dockerfiles when CI is run;
adding a `COPY` to the respective Dockerfiles.

Patching Dockerfiles is not really a "best practice" and, even though the source
code does not and up in the image, the source would still be _sent_ to the daemon
for each build (unless BuildKit is used).

This patch updates the makefiles, circle-ci script, and Dockerfiles;

- When building the Dockerfiles locally, pipe the Dockerfile through stdin.
  Doing so, prevents the build-context from being sent to the daemon. This speeds
  up the build, and doesn't fill up the Docker "temp" directory with content that's
  not used
- Now that no content is sent, add the COPY instructions to the Dockerfiles, and
  remove the code in the circle-ci script to "live patch" the Dockerfiles.

Before this patch is applied (with cache):

```
$ time make -f docker.Makefile build_shell_validate_image
docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck .
Sending build context to Docker daemon     41MB
Step 1/2 : FROM    debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest

2.75 real         0.45 user         0.56 sys
```

After this patch is applied (with cache)::

```
$ time make -f docker.Makefile build_shell_validate_image
cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate -
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM    debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest

0.33 real         0.07 user         0.08 sys
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 166856a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ian Campbell and others added 10 commits June 14, 2019 12:53
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 7c8ee78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With a docker build cache already primed with the build image I am seeing
`time make build -f docker.Makefile DOCKER_BUILDKIT=1 GO_BUILD_CACHE=n` takes
more than 1 minute.

By contrast `time make build -f docker.Makefile DOCKER_BUILDKIT=1
GO_BUILD_CACHE=y` takes less than 10s with a hot cache irrespective of whether
the source tree has changed

Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit d5de835)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.2 (released 2018/11/02) includes fixes to the compiler, linker,
documentation, go command, and the database/sql and go/types packages.

See the milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 58f0bfc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".

See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved

go1.11.3 (released 2018/12/14)

- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230

See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit deaf6e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
See the milestone for details;
https://github.com/golang/go/issues?q=milestone%3AGo1.11.5+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e9d1d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.11.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1500105)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.11.7

Full diff: golang/go@go1.11.6...go1.11.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.10 (released 2019/05/06) includes fixes to the runtime and the linker.
See the Go 1.11.10 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.11.10

Full diff: golang/go@go1.11.9...go1.11.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.11 (released 2019/06/11) includes a fix to the crypto/x509 package.
See the Go 1.11.11 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.11.11

full diff: golang/go@go1.11.10...go1.11.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah force-pushed the 18.09_backport_bump_golang_1.11 branch from c6060b4 to 9bd840b Compare June 14, 2019 10:55
@thaJeztah thaJeztah changed the title [18.09 backport] bump Golang 1.11.10, and some makefile improvements [18.09 backport] bump Golang 1.11.11, and some makefile improvements Jun 14, 2019
@thaJeztah
Copy link
Member Author

rebased, and bumped to Go 1.11.11

@andrewhsu andrewhsu modified the milestones: 18.09.7, 18.09.8 Jun 27, 2019
@thaJeztah thaJeztah modified the milestones: 18.09.8, 18.09.9 Jul 18, 2019
Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kolyshkin kolyshkin merged commit 5f4a501 into docker:18.09 Jul 23, 2019
@thaJeztah thaJeztah deleted the 18.09_backport_bump_golang_1.11 branch July 23, 2019 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants