Skip to content

Commit

Permalink
Squashed 'tools/' changes from 41c5622..d6cc704
Browse files Browse the repository at this point in the history
d6cc704 Fix comment
7139116 Revert "Push comments to the left so they don't appear in scripts"
e47e58f Push comments to the left so they don't appear in scripts
3945fce Remove nonexistent env var GIT_TAG
cd62992 Merge pull request #156 from weaveworks/drop-quay
af0eb51 Merge pull request #157 from weaveworks/fix-image-tag-prefix-length
0b9aee4 Fix image-tag object name prefix length to 8 chars.
813c28f Move from CircleCI 1.0 to 2.0
425cf4e Move from quay.io to Dockerhub
87ccf4f Merge pull request #155 from weaveworks/go-1-12
c31bc28 Update lint script to work with Go 1.12
ed8e380 Update to Go 1.12.1
ec369f5 Merge pull request #153 from dholbach/drop-email
ef7418d weave-users mailing list is closed: https://groups.google.com/a/weave.works/forum/#!topic/weave-users/0QXWGOPdBfY
6954a57 Merge pull request #144 from weaveworks/golang-1.11.1
9649eed Upgrade build image from golang:1.10.0-strech to 1.11.1-strech
59263a7 Merge pull request #141 from weaveworks/update-context
e235c9b Merge pull request #143 from weaveworks/gc-wks-test-vms
c865b4c scheduler: please lint/flake8
da61568 scheduler: please lint/yapf
ce9d78e scheduler: do not cache discovery doc
e4b7873 scheduler: add comment about GCP projects' IAM roles needed to list/delete instances and firewall rules
ff7ec8e scheduler: add comment about CircleCI projects' access via the API
2477d98 scheduler: deploy command now sets the current datetime as the version
5fcd880 scheduler: pass CircleCI API token in for private projects
6b8c323 scheduler: more details in case of failure to get running builds from CircleCI
0871aff scheduler: downgrade google-api-python-client from 1.7.4 to 1.6.7
b631e7f scheduler: add GC of WKS test VMs and firewall rules
a923a32 scheduler: document setup and deployment
013f508 scheduler: lock dependencies' versions
6965a4a Merge pull request #142 from weaveworks/fix-build
23298c6 Fix golint expects import golang.org/x/lint/golint
482f4cd Context is now part of the Go standard library
2bbc9a0 Merge pull request #140 from weaveworks/sched-http-retry
c3726de Add retries to sched util http calls
2cc7b5a Merge pull request #139 from meghalidhoble/master
fd9b0a7 Change : Modified the lint tools to skip the shfmt check if not installed. Why the change : For ppc64le the specific version of shfmt is not available, hence skipped completely the installation of shfmt tool. Thus this change made.
bc645c7 Merge pull request #138 from dholbach/add-license-file
a642e02 license: add Apache 2.0 license text
9bf5956 Merge pull request #109 from hallum/master
d971d82 Merge pull request #134 from weaveworks/2018-07-03-gcloud-regepx
32e7aa2 Merge pull request #137 from weaveworks/gcp-fw-allow-kube-apiserver
bbb6735 Allow CI to access k8s API server on GCP instances
764d46c Merge pull request #135 from weaveworks/2018-07-04-docker-ansible-playbook
ecc2a4e Merge pull request #136 from weaveworks/2018-07-05-gcp-private-ips
209b7fb tools: Add private_ips to the terraform output
369a655 tools: Add an ansible playbook that just installs docker
a643e27 tools: Use --filter instead of --regexp with gcloud
b8eca88 Merge pull request #128 from weaveworks/actually-say-whats-wrong
379ce2b Merge pull request #133 from weaveworks/fix-decrypt
3b906b5 Fix incompatibility with recent versions of OpenSSL
f091ab4 Merge pull request #132 from weaveworks/add-opencontainers-labels-to-dockerfiles
248def1 Inject git revision in Dockerfiles
64f2c28 Add org.opencontainers.image.* labels to Dockerfiles
ea96d8e add information about how to get help (#129)
f066ccd Make yapf diff failure look like an error
34d81d7 Merge pull request #127 from weaveworks/golang-1.10.0-stretch
89a0b4f Use golang:1.10.0-stretch image.
ca69607 Merge pull request #126 from weaveworks/disable-apt-daily-test
f5dc5d5 Create "setup-apt" role
7fab441 Rename bazel to bazel-rules (#125)
ccc8316 Revert "Gocyclo should return error code if issues detected" (#124)
1fe184f Bazel rules for building gogo protobufs (#123)
b917bb8 Merge pull request #122 from weaveworks/fix-scope-gc
c029ce0 Add regex to match scope VMs
0d4824b Merge pull request #121 from weaveworks/provisioning-readme-terraform
5a82d64 Move terraform instructions to tf section
d285d78 Merge pull request #120 from weaveworks/gocyclo-return-value
76b94a4 Do not spawn subshell when reading cyclo output
93b3c0d Use golang:1.9.2-stretch image
d40728f Gocyclo should return error code if issues detected
c4ac1c3 Merge pull request #114 from weaveworks/tune-spell-check
8980656 Only check files
12ebc73 Don't spell-check pki files
578904a Special-case spell-check the same way we do code checks
e772ed5 Special-case on mime type and extension using just patterns
ae82b50 Merge pull request #117 from weaveworks/test-verbose
8943473 Propagate verbose flag to 'go test'.
7c79b43 Merge pull request #113 from weaveworks/update-shfmt-instructions
258ef01 Merge pull request #115 from weaveworks/extra-linting
e690202 Use tools in built image to lint itself
126eb56 Add shellcheck to bring linting in line with scope
63ad68f Don't run lint on files under .git
51d908a Update shfmt instructions
e91cb0d Merge pull request #112 from weaveworks/add-python-lint-tools
0c87554 Add yapf and flake8 to golang build image
35679ee Merge pull request #110 from weaveworks/parallel-push-errors
3ae41b6 Remove unneeded if block
51ff31a Exit on first error
0faad9f Check for errors when pushing images in parallel
d87cd02 Add arg flag override for destination socks host:port in pacfile.
74dc626 Merge pull request #108 from weaveworks/disable-apt-daily
b4f1d91 Merge pull request #107 from weaveworks/docker-17-update
7436aa1 Override apt daily job to not run immediately on boot
7980f15 Merge pull request #106 from weaveworks/document-docker-install-role
f741e53 Bump to Docker 17.06 from CE repo
61796a1 Update Docker CE Debian repo details
0d86f5e Allow for Docker package to be named docker-ce
065c68d Document selection of Docker installation role.
3809053 Just --porcelain; it defaults to v1
11400ea Merge pull request #105 from weaveworks/remove-weaveplugin-remnants
b8b4d64 remove weaveplugin remnants
35099c9 Merge pull request #104 from weaveworks/pull-docker-py
cdd48fc Pull docker-py to speed tests/builds up.
e1c6c24 Merge pull request #103 from weaveworks/test-build-tags
d5d71e0 Add -tags option so callers can pass in build tags
8949b2b Merge pull request #98 from weaveworks/git-status-tag
ac30687 Merge pull request #100 from weaveworks/python_linting
4b125b5 Pin yapf & flake8 versions
7efb485 Lint python linting function
444755b Swap diff direction to reflect changes required
c5b2434 Install flake8 & yapf
5600eac Lint python in build-tools repo
0b02ca9 Add python linting
c011c0d Merge pull request #79 from kinvolk/schu/python-shebang
6577d07 Merge pull request #99 from weaveworks/shfmt-version
00ce0dc Use git status instead of diff to add 'WIP' tag
411fd13 Use shfmt v1.3.0 instead of latest from master.
0d6d4da Run shfmt 1.3 on the code.
5cdba32 Add sudo
c322ca8 circle.yml: Install shfmt binary.
e59c225 Install shfmt 1.3 binary.
30706e6 Install pyhcl in the build container.
960d222 Merge pull request #97 from kinvolk/alban/update-shfmt-3
1d535c7 shellcheck: fix escaping issue
5542498 Merge pull request #96 from kinvolk/alban/update-shfmt-2
32f7cc5 shfmt: fix coding style
09f72af lint: print the diff in case of error
571c7d7 Merge pull request #95 from kinvolk/alban/update-shfmt
bead6ed Update for latest shfmt
b08dc4d Update for latest shfmt (#94)
2ed8aaa Add no-race argument to test script (#92)
80dd78e Merge pull request #91 from weaveworks/upgrade-go-1.8.1
08dcd0d Please ./lint as shfmt changed its rules between 1.0.0 and 1.3.0.
a8bc9ab Upgrade default Go version to 1.8.1.
31d069d Change Python shebang to `#!/usr/bin/env python`

git-subtree-dir: tools
git-subtree-split: d6cc704a2892e8d85aa8fa4d201c1a404f02dfa4
  • Loading branch information
bboreham committed May 15, 2019
1 parent 50bc80f commit 57a64ff
Show file tree
Hide file tree
Showing 46 changed files with 1,237 additions and 578 deletions.
36 changes: 36 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
version: 2
jobs:
build:
# Use 'machine' type so we can run the lint step with directory mounted
machine:
docker_layer_caching: true
working_directory: /home/circleci/src/github.com/weaveworks/build-tools
environment:
GOPATH: /home/circleci/
steps:
- checkout
- run: cd build; make
- run: docker run --rm -v "$PWD:$PWD" -w "$PWD" --entrypoint sh weaveworks/build-golang -c ./lint .
- run: cd cover; make
# Socks makefile needs to overwrite Go std library
- run: sudo chmod a+wr --recursive /usr/local/go/pkg
- run: cd socks; make
- run: cd runner; make

- deploy:
command: |
if [ "${CIRCLE_BRANCH}" == "master" ]; then
cd build
docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
for image in $(make images); do
# Push all tags - latest and git-tag
docker push "${image}"
# Tag the built images with something derived from the base images in
# their respective Dockerfiles. So "FROM golang:1.8.0-stretch" as a
# base image would lead to a tag of "1.8.0-stretch"
IMG_TAG=$(make "imagetag-${image#weaveworks/build-}")
docker tag "${image}:latest" "${image}:${IMG_TAG}"
docker push "${image}:${IMG_TAG}"
done
fi
175 changes: 175 additions & 0 deletions COPYING.LGPL-3
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
./integration/assert.sh is a copy of

https://github.com/lehmannro/assert.sh/blob/master/assert.sh

Since it was added to this codebase, it has only received cosmetic
modifications. As it is licensed under the LGPL-3, here's the license
text in its entirety:



GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.


This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.

0. Additional Definitions.

As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.

"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.

An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.

A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".

The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.

The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.

1. Exception to Section 3 of the GNU GPL.

You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.

2. Conveying Modified Versions.

If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:

a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or

b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.

3. Object Code Incorporating Material from Library Header Files.

The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:

a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.

b) Accompany the object code with a copy of the GNU GPL and this license
document.

4. Combined Works.

You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:

a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.

b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.

c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.

d) Do one of the following:

0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.

1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.

e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)

5. Combined Libraries.

You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:

a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.

b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.

6. Revised Versions of the GNU Lesser General Public License.

The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.

Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.

If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
13 changes: 13 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Copyright 2018 Weaveworks. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
17 changes: 16 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

Included in this repo are tools shared by weave.git and scope.git. They include

- ```bazel-rules```: Bazel build rules used in our projects
- ```build```: a set of docker base-images for building weave
projects. These should be used instead of giving each project its
own build image.
Expand Down Expand Up @@ -32,7 +33,11 @@ Included in this repo are tools shared by weave.git and scope.git. They include
## Requirements

- ```lint``` requires shfmt to lint sh files; get shfmt with
```go get -u gopkg.in/mvdan/sh.v1/cmd/shfmt```
```
curl -fsSLo shfmt https://github.com/mvdan/sh/releases/download/v1.3.0/shfmt_v1.3.0_linux_amd64
chmod +x shfmt
```
(we pin that version, and it doesn't build from the source repo any more)

## Using build-tools.git

Expand All @@ -50,3 +55,13 @@ To update the code in build-tools.git, the process is therefore:
- PR into build-tools.git, go through normal review process etc.
- Do `git subtree pull --prefix tools https://github.com/weaveworks/build-tools.git master --squash`
in your repo, and PR that.
## <a name="help"></a>Getting Help
If you have any questions about, feedback for or problems with `build-tools`:
- Invite yourself to the <a href="https://slack.weave.works/" target="_blank">Weave Users Slack</a>.
- Ask a question on the [#general](https://weave-community.slack.com/messages/general/) slack channel.
- [File an issue](https://github.com/weaveworks/build-tools/issues/new).
Your feedback is always welcome!
26 changes: 26 additions & 0 deletions bazel-rules/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
load("@io_bazel_rules_go//proto:compiler.bzl", "go_proto_compiler")

go_proto_compiler(
name = "gogo_proto",
deps = [
"//vendor/github.com/gogo/protobuf/gogoproto:go_default_library",
"//vendor/github.com/gogo/protobuf/proto:go_default_library",
"//vendor/github.com/gogo/protobuf/sortkeys:go_default_library",
],
plugin = "@com_github_gogo_protobuf//protoc-gen-gogoslick",
visibility = ["//visibility:public"],
)

go_proto_compiler(
name = "gogo_grpc",
deps = [
"//vendor/github.com/gogo/protobuf/gogoproto:go_default_library",
"//vendor/github.com/gogo/protobuf/proto:go_default_library",
"//vendor/github.com/gogo/protobuf/sortkeys:go_default_library",
"//vendor/google.golang.org/grpc:go_default_library",
"//vendor/golang.org/x/net/context:go_default_library",
],
plugin = "@com_github_gogo_protobuf//protoc-gen-gogoslick",
options = ["plugins=grpc"],
visibility = ["//visibility:public"],
)
36 changes: 36 additions & 0 deletions bazel-rules/gogo.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
load("@io_bazel_rules_go//go:def.bzl", "go_repository")


_BUILD_FILE = """
proto_library(
name = "gogoproto",
srcs = ["gogo.proto"],
deps = [
"@com_google_protobuf//:descriptor_proto",
],
visibility = ["//visibility:public"],
)
"""

def _go_repository_impl(ctx):
ctx.file("BUILD.bazel", content="")
ctx.file("github.com/gogo/protobuf/gogoproto/BUILD.bazel", content=_BUILD_FILE)
ctx.template("github.com/gogo/protobuf/gogoproto/gogo.proto", ctx.attr._proto)

_gogo_proto_repository = repository_rule(
implementation = _go_repository_impl,
attrs = {
"_proto": attr.label(default="//vendor/github.com/gogo/protobuf/gogoproto:gogo.proto"),
},
)

def gogo_dependencies():
go_repository(
name = "com_github_gogo_protobuf",
importpath = "github.com/gogo/protobuf",
urls = ["https://codeload.github.com/ianthehat/protobuf/zip/2adc21fd136931e0388e278825291678e1d98309"],
strip_prefix = "protobuf-2adc21fd136931e0388e278825291678e1d98309",
type = "zip",
build_file_proto_mode="disable",
)
_gogo_proto_repository(name = "internal_gogo_proto_repository")
5 changes: 3 additions & 2 deletions build/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@

# Boiler plate for bulding Docker containers.
# All this must go at top of file I'm afraid.
IMAGE_PREFIX := quay.io/weaveworks/build-
IMAGE_PREFIX := weaveworks/build-
IMAGE_TAG := $(shell ../image-tag)
GIT_REVISION := $(shell git rev-parse HEAD)
UPTODATE := .uptodate

# Every directory with a Dockerfile in it builds an image called
# $(IMAGE_PREFIX)<dirname>. Dependencies (i.e. things that go in the image)
# still need to be explicitly declared.
%/$(UPTODATE): %/Dockerfile %/*
$(SUDO) docker build -t $(IMAGE_PREFIX)$(shell basename $(@D)) $(@D)/
$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -t $(IMAGE_PREFIX)$(shell basename $(@D)) $(@D)/
$(SUDO) docker tag $(IMAGE_PREFIX)$(shell basename $(@D)) $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG)
touch $@

Expand Down
19 changes: 15 additions & 4 deletions build/golang/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.8.0-stretch
FROM golang:1.12.1-stretch
RUN apt-get update && \
apt-get install -y \
curl \
Expand All @@ -11,9 +11,14 @@ RUN apt-get update && \
python-pip \
python-requests \
python-yaml \
shellcheck \
unzip && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN pip install attrs
RUN pip install attrs pyhcl yapf==0.16.2 flake8==3.3.0
RUN curl -fsSLo shfmt https://github.com/mvdan/sh/releases/download/v1.3.0/shfmt_v1.3.0_linux_amd64 && \
echo "b1925c2c405458811f0c227266402cf1868b4de529f114722c2e3a5af4ac7bb2 shfmt" | sha256sum -c && \
chmod +x shfmt && \
mv shfmt /usr/bin
RUN go clean -i net && \
go install -tags netgo std && \
go install -race -tags netgo std
Expand All @@ -25,11 +30,10 @@ RUN go get -tags netgo \
github.com/gogo/protobuf/gogoproto \
github.com/gogo/protobuf/protoc-gen-gogoslick \
github.com/golang/dep/... \
github.com/golang/lint/golint \
golang.org/x/lint/golint \
github.com/golang/protobuf/protoc-gen-go \
github.com/kisielk/errcheck \
github.com/mjibson/esc \
github.com/mvdan/sh/cmd/shfmt \
github.com/prometheus/prometheus/cmd/promtool && \
rm -rf /go/pkg /go/src
RUN mkdir protoc && \
Expand All @@ -44,3 +48,10 @@ RUN mkdir -p /var/run/secrets/kubernetes.io/serviceaccount && \
touch /var/run/secrets/kubernetes.io/serviceaccount/token
COPY build.sh /
ENTRYPOINT ["/build.sh"]

ARG revision
LABEL maintainer="Weaveworks <help@weave.works>" \
org.opencontainers.image.title="golang" \
org.opencontainers.image.source="https://github.com/weaveworks/build-tools/tree/master/build/golang" \
org.opencontainers.image.revision="${revision}" \
org.opencontainers.image.vendor="Weaveworks"
Loading

0 comments on commit 57a64ff

Please sign in to comment.