Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🌱 Add licence-scan for pull requests #9184

Merged
merged 1 commit into from
Aug 30, 2023
Merged

🌱 Add licence-scan for pull requests #9184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
- [Features and bugs](#features-and-bugs)
- [Experiments](#experiments)
- [Breaking Changes](#breaking-changes)
- [Dependency Licence Management](#dependency-licence-management)
- [API conventions](#api-conventions)
- [Optional vs. Required](#optional-vs-required)
- [Example](#example)
Expand Down Expand Up @@ -415,6 +416,10 @@ There may, at times, need to be exceptions where breaking changes are allowed in
discretion of the project's maintainers, and must be carefully considered before merging. An example of an allowed
breaking change might be a fix for a behavioral bug that was released in an initial minor version (such as `v0.3.0`).

## Dependency Licence Management

Cluster API follows the [license policy of the CNCF](https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md). This sets limits on which
sbueringer marked this conversation as resolved.
Show resolved Hide resolved
licenses dependencies and other artifacts use. For go dependencies only dependencies listed in the `go.mod` are considered dependencies. This is in line with [how dependencies are reviewed in Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/vendor.md#reviewing-and-approving-dependency-changes).

## API conventions

Expand Down
10 changes: 8 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ HADOLINT_FAILURE_THRESHOLD = warning

SHELLCHECK_VER := v0.9.0

TRIVY_VER := 0.44.1

KPROMO_VER := v4.0.4
KPROMO_BIN := kpromo
KPROMO := $(abspath $(TOOLS_BIN_DIR)/$(KPROMO_BIN)-$(KPROMO_VER))
Expand Down Expand Up @@ -605,7 +607,7 @@ APIDIFF_OLD_COMMIT ?= $(shell git rev-parse origin/main)
apidiff: $(GO_APIDIFF) ## Check for API differences
$(GO_APIDIFF) $(APIDIFF_OLD_COMMIT) --print-compatible

ALL_VERIFY_CHECKS = boilerplate shellcheck tiltfile modules gen conversions doctoc capi-book-summary
ALL_VERIFY_CHECKS = licenses boilerplate shellcheck tiltfile modules gen conversions doctoc capi-book-summary

.PHONY: verify
verify: $(addprefix verify-,$(ALL_VERIFY_CHECKS)) lint-dockerfiles ## Run all verify-* targets
Expand Down Expand Up @@ -657,7 +659,11 @@ verify-tiltfile: ## Verify Tiltfile format

.PHONY: verify-container-images
verify-container-images: ## Verify container images
TRACE=$(TRACE) ./hack/verify-container-images.sh
TRACE=$(TRACE) ./hack/verify-container-images.sh $(TRIVY_VER)

.PHONY: verify-licenses
verify-licenses: ## Verify licenses
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
verify-licenses: ## Verify licenses
verify-licenses: $(TRIVY) ## Verify licenses

Should we start running ensure-trivy.sh as a separate target to fetch the binary (including the versioned binary + symlink to the version) as we do for go install based tools?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I could also take this as a follow-up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure given it's just a download - I'm fine either way though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be a nice follow-up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I reviewed the PR. I'm fine with the current version. I think the most important part is that we always use the right version specified in the Makefile (which we do)

Would be still nice to follow-up for consistency though

TRACE=$(TRACE) ./hack/verify-licenses.sh $(TRIVY_VER)
sbueringer marked this conversation as resolved.
Show resolved Hide resolved

.PHONY: verify-govulncheck
verify-govulncheck: $(GOVULNCHECK) ## Verify code for vulnerabilities
Expand Down
57 changes: 57 additions & 0 deletions hack/ensure-trivy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

VERSION=${1}

GO_OS="$(go env GOOS)"
if [[ "${GO_OS}" == "linux" ]]; then
TRIVY_OS="Linux"
elif [[ "${GO_OS}" == "darwin"* ]]; then
TRIVY_OS="macOS"
fi

GO_ARCH="$(go env GOARCH)"
if [[ "${GO_ARCH}" == "amd" ]]; then
TRIVY_ARCH="32bit"
elif [[ "${GO_ARCH}" == "amd64"* ]]; then
TRIVY_ARCH="64bit"
elif [[ "${GO_ARCH}" == "arm" ]]; then
TRIVY_ARCH="ARM"
elif [[ "${GO_ARCH}" == "arm64" ]]; then
TRIVY_ARCH="ARM64"
fi

TOOL_BIN=hack/tools/bin
mkdir -p ${TOOL_BIN}

TRIVY="${TOOL_BIN}/trivy/${VERSION}/trivy"

# Downloads trivy scanner
if [ ! -f "$TRIVY" ]; then
curl -L -o ${TOOL_BIN}/trivy.tar.gz "https://github.com/aquasecurity/trivy/releases/download/v${VERSION}/trivy_${VERSION}_${TRIVY_OS}-${TRIVY_ARCH}.tar.gz"
mkdir -p "$(dirname "$0")/tools/bin/trivy/${VERSION}"
killianmuldoon marked this conversation as resolved.
Show resolved Hide resolved
tar -xf "${TOOL_BIN}/trivy.tar.gz" -C "${TOOL_BIN}/trivy/${VERSION}" trivy
chmod +x "${TOOL_BIN}/trivy/${VERSION}/trivy"
rm "${TOOL_BIN}/trivy.tar.gz"
fi
45 changes: 11 additions & 34 deletions hack/verify-container-images.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,48 +22,25 @@ if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

TRIVY_VERSION=0.34.0
VERSION=${1}

Copy link
Member

@sbueringer sbueringer Aug 30, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@killianmuldoon Found in https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/pull/2299/files. We probably still want to set GO_ARCH

GO_OS="$(go env GOOS)"
if [[ "${GO_OS}" == "linux" ]]; then
TRIVY_OS="Linux"
elif [[ "${GO_OS}" == "darwin"* ]]; then
TRIVY_OS="macOS"
fi

GO_ARCH="$(go env GOARCH)"
if [[ "${GO_ARCH}" == "amd" ]]; then
TRIVY_ARCH="32bit"
elif [[ "${GO_ARCH}" == "amd64"* ]]; then
TRIVY_ARCH="64bit"
elif [[ "${GO_ARCH}" == "arm" ]]; then
TRIVY_ARCH="ARM"
elif [[ "${GO_ARCH}" == "arm64" ]]; then
TRIVY_ARCH="ARM64"
fi

TOOL_BIN=hack/tools/bin
mkdir -p ${TOOL_BIN}

# Downloads trivy scanner
curl -L -o ${TOOL_BIN}/trivy.tar.gz "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_${TRIVY_OS}-${TRIVY_ARCH}.tar.gz"
REPO_ROOT=$(git rev-parse --show-toplevel)
"${REPO_ROOT}/hack/ensure-trivy.sh" "${VERSION}"

tar -xf "${TOOL_BIN}/trivy.tar.gz" -C "${TOOL_BIN}" trivy
chmod +x ${TOOL_BIN}/trivy
rm ${TOOL_BIN}/trivy.tar.gz
TRIVY="${REPO_ROOT}/hack/tools/bin/trivy/${VERSION}/trivy"

# Builds all the container images to be scanned and cleans up changes to ./*manager_image_patch.yaml ./*manager_pull_policy.yaml.
make REGISTRY=gcr.io/k8s-staging-cluster-api PULL_POLICY=IfNotPresent TAG=dev docker-build
make clean-release-git

# Scan the images
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/clusterctl-"${GO_ARCH}":dev && R1=$? || R1=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/test-extension-"${GO_ARCH}":dev && R2=$? || R2=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/kubeadm-control-plane-controller-"${GO_ARCH}":dev && R3=$? || R3=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/kubeadm-bootstrap-controller-"${GO_ARCH}":dev && R4=$? || R4=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/cluster-api-controller-"${GO_ARCH}":dev && R5=$? || R5=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/capd-manager-"${GO_ARCH}":dev && R6=$? || R6=$?
${TOOL_BIN}/trivy image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/capim-manager-"${GO_ARCH}":dev && R6=$? || R6=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/clusterctl-"${GO_ARCH}":dev && R1=$? || R1=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/test-extension-"${GO_ARCH}":dev && R2=$? || R2=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/kubeadm-control-plane-controller-"${GO_ARCH}":dev && R3=$? || R3=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/kubeadm-bootstrap-controller-"${GO_ARCH}":dev && R4=$? || R4=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/cluster-api-controller-"${GO_ARCH}":dev && R5=$? || R5=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/capd-manager-"${GO_ARCH}":dev && R6=$? || R6=$?
"${TRIVY}" image -q --exit-code 1 --ignore-unfixed --severity MEDIUM,HIGH,CRITICAL gcr.io/k8s-staging-cluster-api/capim-manager-"${GO_ARCH}":dev && R6=$? || R6=$?

echo ""
BRed='\033[1;31m'
Expand Down
41 changes: 41 additions & 0 deletions hack/verify-licenses.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi

# This list is from https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md
CNCF_LICENSE_ALLOWLIST=Apache-2.0,MIT,BSD-2-Clause,SD-2-Clause-FreeBSD,BSD-3-Clause,ISC,Python-2.0,PostgreSQL,X11,Zlib

VERSION=${1}

REPO_ROOT=$(git rev-parse --show-toplevel)
"${REPO_ROOT}/hack/ensure-trivy.sh" "${VERSION}"


TRIVY="${REPO_ROOT}/hack/tools/bin/trivy/${VERSION}/trivy"
sbueringer marked this conversation as resolved.
Show resolved Hide resolved
$TRIVY filesystem . --license-full --ignored-licenses ${CNCF_LICENSE_ALLOWLIST} --scanners license --severity UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL -f json | \
sbueringer marked this conversation as resolved.
Show resolved Hide resolved
# Specifically ignore 'github.com/hashicorp/hcl'. This is a known indirect dependency that we should remove where possible.
# This query ensures we only skip github.com/hashicorp/hcl for as long as the license remains MPL-2.0
jq '.Results[] | select( .Licenses[]?.PkgName == "github.com/hashicorp/hcl" and .Licenses[]?.Name == "MPL-2.0" | not) | if . == {} then . else error(.) end'
sbueringer marked this conversation as resolved.
Show resolved Hide resolved