Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Add license scan for pull requests #2299

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,12 @@ Kubernetes projects require that you sign a Contributor License Agreement (CLA)
* 📖 (:book:, documentation or proposals)
* 🌱 (:seedling:, minor or other)

### Contributer Ladder
## Dependency Licence Management

Cluster API provider vSphere follows the [license policy of the CNCF](https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md). This sets limits on which
licenses dependencies and other artifacts use. For go dependencies only dependencies listed in the `go.mod` are considered dependencies. This is in line with [how dependencies are reviewed in Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/vendor.md#reviewing-and-approving-dependency-changes).

### Contributor Ladder

We broadly follow the requirements from the [Kubernetes Community Membership](https://github.com/kubernetes/community/blob/master/community-membership.md).

Expand Down
10 changes: 8 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ GO_APIDIFF_PKG := github.com/joelanford/go-apidiff

SHELLCHECK_VER := v0.9.0

TRIVY_VER := 0.44.1

KPROMO_VER := v4.0.4
KPROMO_BIN := kpromo
KPROMO := $(abspath $(TOOLS_BIN_DIR)/$(KPROMO_BIN)-$(KPROMO_VER))
Expand Down Expand Up @@ -334,7 +336,7 @@ APIDIFF_OLD_COMMIT ?= $(shell git rev-parse origin/main)
apidiff: $(GO_APIDIFF) ## Check for API differences
$(GO_APIDIFF) $(APIDIFF_OLD_COMMIT) --print-compatible

ALL_VERIFY_CHECKS = boilerplate shellcheck modules gen conversions doctoc flavors
ALL_VERIFY_CHECKS = licenses boilerplate shellcheck modules gen conversions doctoc flavors

.PHONY: verify
verify: $(addprefix verify-,$(ALL_VERIFY_CHECKS)) ## Run all verify-* targets
Expand Down Expand Up @@ -378,7 +380,11 @@ verify-shellcheck: ## Verify shell files

.PHONY: verify-container-images
verify-container-images: ## Verify container images
TRACE=$(TRACE) ./hack/verify-container-images.sh
TRACE=$(TRACE) ./hack/verify-container-images.sh $(TRIVY_VER)

.PHONY: verify-licenses
verify-licenses: ## Verify licenses
TRACE=$(TRACE) ./hack/verify-licenses.sh $(TRIVY_VER)

.PHONY: verify-govulncheck
verify-govulncheck: $(GOVULNCHECK) ## Verify code for vulnerabilities
Expand Down
57 changes: 57 additions & 0 deletions hack/ensure-trivy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

VERSION=${1}

GO_OS="$(go env GOOS)"
if [[ "${GO_OS}" == "linux" ]]; then
TRIVY_OS="Linux"
elif [[ "${GO_OS}" == "darwin"* ]]; then
TRIVY_OS="macOS"
fi

GO_ARCH="$(go env GOARCH)"
if [[ "${GO_ARCH}" == "amd" ]]; then
TRIVY_ARCH="32bit"
elif [[ "${GO_ARCH}" == "amd64"* ]]; then
TRIVY_ARCH="64bit"
elif [[ "${GO_ARCH}" == "arm" ]]; then
TRIVY_ARCH="ARM"
elif [[ "${GO_ARCH}" == "arm64" ]]; then
TRIVY_ARCH="ARM64"
fi

TOOL_BIN=hack/tools/bin
mkdir -p ${TOOL_BIN}

TRIVY="${TOOL_BIN}/trivy/${VERSION}/trivy"

# Downloads trivy scanner
if [ ! -f "$TRIVY" ]; then
curl -L -o ${TOOL_BIN}/trivy.tar.gz "https://github.com/aquasecurity/trivy/releases/download/v${VERSION}/trivy_${VERSION}_${TRIVY_OS}-${TRIVY_ARCH}.tar.gz"
mkdir -p "$(dirname "$0")/tools/bin/trivy/${VERSION}"
tar -xf "${TOOL_BIN}/trivy.tar.gz" -C "${TOOL_BIN}/trivy/${VERSION}" trivy
chmod +x "${TOOL_BIN}/trivy/${VERSION}/trivy"
rm "${TOOL_BIN}/trivy.tar.gz"
fi
34 changes: 6 additions & 28 deletions hack/verify-container-images.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
# Copyright 2022 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -22,42 +22,20 @@ if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

TRIVY_VERSION=0.34.0

GO_OS="$(go env GOOS)"
if [[ "${GO_OS}" == "linux" ]]; then
TRIVY_OS="Linux"
elif [[ "${GO_OS}" == "darwin"* ]]; then
TRIVY_OS="macOS"
fi

VERSION=${1}
GO_ARCH="$(go env GOARCH)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting that we don't have this line in core CAPI and probably just assume that GOARCH is set?
Maybe something to fix in core CAPI? (@killianmuldoon)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GO_ARCH here is only used to give the resulting image a name - I'm not sure if it's used for anything else at this point - the ensure-trivy script sets its own GO_ARCH.

Maybe it's a good idea to just remove this here?

Copy link
Member

@sbueringer sbueringer Aug 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not used to give it a name, it's used to match the name that is used by make docker-build.

I think if we drop this line we either:

  • have to pass GO_ARCH into the script
  • have to find another way to match it correctly (I guess GOARCH might be different on macOS vs Linux (?))

I would avoid implicitly depending on an env variable being exported by someone calling that script (which I think is what is happening in core CAPI now). (Because this makes it harder to call the script standalone and just a bit too magical/implicit for my taste)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right - I'll do a PR into CAPI to make them consistent and make CAPI's use of the GO_ARCH more explicit.

if [[ "${GO_ARCH}" == "amd" ]]; then
TRIVY_ARCH="32bit"
elif [[ "${GO_ARCH}" == "amd64"* ]]; then
TRIVY_ARCH="64bit"
elif [[ "${GO_ARCH}" == "arm" ]]; then
TRIVY_ARCH="ARM"
elif [[ "${GO_ARCH}" == "arm64" ]]; then
TRIVY_ARCH="ARM64"
fi

TOOL_BIN=hack/tools/bin
mkdir -p ${TOOL_BIN}

# Downloads trivy scanner
curl -L -o ${TOOL_BIN}/trivy.tar.gz "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_${TRIVY_OS}-${TRIVY_ARCH}.tar.gz"
REPO_ROOT=$(git rev-parse --show-toplevel)
"${REPO_ROOT}/hack/ensure-trivy.sh" "${VERSION}"

tar -xf "${TOOL_BIN}/trivy.tar.gz" -C "${TOOL_BIN}" trivy
chmod +x ${TOOL_BIN}/trivy
rm ${TOOL_BIN}/trivy.tar.gz
TRIVY="${REPO_ROOT}/hack/tools/bin/trivy/${VERSION}/trivy"

# Builds all the container images to be scanned and cleans up changes to ./*manager_image_patch.yaml ./*manager_pull_policy.yaml.
make REGISTRY=gcr.io/k8s-staging-capi-vsphere PULL_POLICY=IfNotPresent TAG=dev docker-build
make clean-release-git

# Scan the images
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-capi-vsphere/cluster-api-vsphere-controller-"${GO_ARCH}":dev && R1=$? || R1=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-capi-vsphere/cluster-api-vsphere-controller-"${GO_ARCH}":dev && R1=$? || R1=$?

echo ""
BRed='\033[1;31m'
Expand Down
38 changes: 38 additions & 0 deletions hack/verify-licenses.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

# This list is from https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md
CNCF_LICENSE_ALLOWLIST=Apache-2.0,MIT,BSD-2-Clause,SD-2-Clause-FreeBSD,BSD-3-Clause,ISC,Python-2.0,PostgreSQL,X11,Zlib

VERSION=${1}

REPO_ROOT=$(git rev-parse --show-toplevel)
"${REPO_ROOT}/hack/ensure-trivy.sh" "${VERSION}"


TRIVY="${REPO_ROOT}/hack/tools/bin/trivy/${VERSION}/trivy"
$TRIVY filesystem . --license-full --ignored-licenses ${CNCF_LICENSE_ALLOWLIST} --scanners license --severity UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL -f json | \
# Specifically ignore 'github.com/hashicorp/hcl'. This is a known indirect dependency that we should remove where possible.
# This query ensures we only skip github.com/hashicorp/hcl for as long as the license remains MPL-2.0
jq '.Results[] | select( .Licenses[]?.PkgName == "github.com/hashicorp/hcl" and .Licenses[]?.Name == "MPL-2.0" | not) | if . == {} then . else error(.) end'