Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build: Make distroless image default and publish mimir-alpine #8204

Merged
merged 4 commits into from
May 30, 2024
Merged

Build: Make distroless image default and publish mimir-alpine #8204

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
836115e
Security: Make distroless image default and publish mimir-alpine
ying-jeanne May 28, 2024
fca8c2c
Update CHANGELOG.md
ying-jeanne May 28, 2024
18ec3e8
Update CHANGELOG.md
ying-jeanne May 30, 2024
3fb8c2a
remove extra_package for distroless uptodate_race target
ying-jeanne May 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/scripts/build-images.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ do
make \
BUILD_IN_CONTAINER=false \
PUSH_MULTIARCH_TARGET="type=oci,dest=$OUTPUT/$NAME.oci" \
PUSH_MULTIARCH_TARGET_DISTROLESS="type=oci,dest=$OUTPUT/$NAME\-distroless.oci" \
PUSH_MULTIARCH_TARGET_ALPINE="type=oci,dest=$OUTPUT/$NAME\-alpine.oci" \
PUSH_MULTIARCH_TARGET_CONTINUOUS_TEST="type=oci,dest=$OUTPUT/$NAME\-continuous\-test.oci" \
push-multiarch-$target
done
16 changes: 8 additions & 8 deletions .github/workflows/test-build-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,15 +281,15 @@ jobs:
- name: Build Mimir with race-detector
run: |
# When building uptodate_race target, we create two images since
# a Dockerfile.distroless exists. The distroless image is built with
# the `-distroless` suffix.
# a Dockerfile.alpine exists. The alpine image is built with
# the `-alpine` suffix.
# We build both until we have finished migrating to distroless.
# We test the distroless race image in every integration test.
# We test the (legacy) alpine race image when deploying (aka pushing).
make BUILD_IN_CONTAINER=false cmd/mimir/.uptodate_race
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_DISTROLESS_IMAGE="grafana/mimir-distroless:$IMAGE_TAG_RACE"
export MIMIR_ALPINE_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_DISTROLESS_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_ALPINE_IMAGE="grafana/mimir-alpine:$IMAGE_TAG_RACE"
docker save $MIMIR_DISTROLESS_IMAGE -o ./mimir_race_image_distroless
docker save $MIMIR_ALPINE_IMAGE -o ./mimir_race_image_alpine
- name: Upload archive with race-enabled Mimir
Expand Down Expand Up @@ -355,7 +355,7 @@ jobs:
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
docker load -i ./mimir_race_image_distroless
docker run "grafana/mimir-distroless:$IMAGE_TAG_RACE" --version
docker run "grafana/mimir:$IMAGE_TAG_RACE" --version
- name: Preload Images
# We download docker images used by integration tests so that all images are available
# locally and the download time doesn't account in the test execution time, which is subject
Expand All @@ -364,7 +364,7 @@ jobs:
- name: Integration Tests
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_IMAGE="grafana/mimir-distroless:$IMAGE_TAG_RACE"
export MIMIR_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export IMAGE_TAG=$(make image-tag)
export MIMIRTOOL_IMAGE="grafana/mimirtool:$IMAGE_TAG"
export MIMIR_CHECKOUT_DIR="/go/src/github.com/grafana/mimir"
Expand Down Expand Up @@ -428,7 +428,7 @@ jobs:
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
docker load -i ./mimir_race_image_alpine
docker run "grafana/mimir:$IMAGE_TAG_RACE" --version
docker run "grafana/mimir-alpine:$IMAGE_TAG_RACE" --version
- name: Preload Images
# We download docker images used by integration tests so that all images are available
# locally and the download time doesn't account in the test execution time, which is subject
Expand All @@ -437,7 +437,7 @@ jobs:
- name: Integration Tests
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_IMAGE="grafana/mimir-alpine:$IMAGE_TAG_RACE"
export IMAGE_TAG=$(make image-tag)
export MIMIRTOOL_IMAGE="grafana/mimirtool:$IMAGE_TAG"
export MIMIR_CHECKOUT_DIR="/go/src/github.com/grafana/mimir"
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### Grafana Mimir

* [CHANGE] Build: Publish distroless as grafana/mimir image by default. #8204
ying-jeanne marked this conversation as resolved.
Show resolved Hide resolved
* [CHANGE] Ingester: `/ingester/flush` endpoint is now only allowed to execute only while the ingester is in `Running` state. The 503 status code is returned if the endpoint is called while the ingester is not in `Running` state. #7486
* [CHANGE] Distributor: Include label name in `err-mimir-label-value-too-long` error message: #7740
* [CHANGE] Ingester: enabled 1 out 10 errors log sampling by default. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. The feature can be configured via `-ingester.error-sample-rate` (0 to log all errors). #7807
Expand Down
54 changes: 34 additions & 20 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,11 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-continuous-test:$(IMAGE_TAG) $(@D)/; \
fi;
if [ -f $(@D)/Dockerfile.distroless ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.distroless \
if [ -f $(@D)/Dockerfile.alpine ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.alpine \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG) $(@D)/; \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG) $(@D)/; \
fi;
@echo
$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)$(shell basename $(@D)) -t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG) $(@D)/
Expand All @@ -117,7 +117,7 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-continuous-test:$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG)
@echo
@echo Please use '"make push-multiarch-build-image"' to build and push build image.
@echo Please use '"make push-multiarch-mimir"' to build and push Mimir image.
Expand All @@ -126,32 +126,40 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)

%/$(UPTODATE_RACE): GOOS=linux
%/$(UPTODATE_RACE): %/Dockerfile
# Build Dockerfile.distroless if it exists, we use distroless/base-nossl-debian12 as base image since race detector needs glibc packages.
if [ -f $(@D)/Dockerfile.distroless ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.distroless \
# Build Dockerfile.alpine if it exists
if [ -f $(@D)/Dockerfile.alpine ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.alpine \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
--build-arg=BINARY_SUFFIX=_race \
--build-arg=BASEIMG="gcr.io/distroless/base-nossl-debian12" \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG_RACE) $(@D)/; \
--build-arg=EXTRA_PACKAGES="gcompat" \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG_RACE) $(@D)/; \
fi;
@echo
# We need gcompat -- compatibility layer with glibc, as race-detector currently requires glibc, but Alpine uses musl libc instead.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment should be moved to "alpine" version. For distroless image, we should restore comment about using distroless/base-nossl-debian12 as base image.

$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) --build-arg=USE_BINARY_SUFFIX=true --build-arg=BINARY_SUFFIX=_race --build-arg=EXTRA_PACKAGES="gcompat" -t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE) $(@D)/
# for grafana/mimir which is based on distroless image, the EXTRA_PACKAGES is just ignored.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we even pass EXTRA_PACKAGES to the build then? We can remove it.

Copy link
Contributor Author

@ying-jeanne ying-jeanne May 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a search in project, this is used in other dockerfiles still, not only for cmd/mimir, we still need it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean now, it is UPTODATE_RACE target which is only for cmd/mimir, removing it.

$(SUDO) docker build \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
--build-arg=BINARY_SUFFIX=_race \
--build-arg=EXTRA_PACKAGES="gcompat" \
--build-arg=BASEIMG="gcr.io/distroless/base-nossl-debian12" \
-t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE) $(@D)/
@echo
@echo Go binaries were built using GOOS=$(GOOS) and GOARCH=$(GOARCH)
@echo
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG_RACE)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG_RACE)
@echo
@touch $@

# This variable controls where result of building of multiarch image should be sent. Default is registry.
# Other options are documented in https://docs.docker.com/engine/reference/commandline/buildx_build/#output.
# CI workflow uses PUSH_MULTIARCH_TARGET="type=oci,dest=file.oci" to store images locally for next steps in the pipeline.
PUSH_MULTIARCH_TARGET ?= type=registry
PUSH_MULTIARCH_TARGET_DISTROLESS ?= type=registry
PUSH_MULTIARCH_TARGET_ALPINE ?= type=registry
PUSH_MULTIARCH_TARGET_CONTINUOUS_TEST ?= type=registry

# This target compiles mimir for linux/amd64 and linux/arm64 and then builds and pushes a multiarch image to the target repository.
Expand All @@ -164,7 +172,13 @@ push-multiarch-%/$(UPTODATE):
$(MAKE) GOOS=linux GOARCH=amd64 BINARY_SUFFIX=_linux_amd64 $(DIR)/$(shell basename $(DIR)); \
$(MAKE) GOOS=linux GOARCH=arm64 BINARY_SUFFIX=_linux_arm64 $(DIR)/$(shell basename $(DIR)); \
fi
$(SUDO) docker buildx build -o $(PUSH_MULTIARCH_TARGET) --platform linux/amd64,linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) --build-arg=USE_BINARY_SUFFIX=true -t $(IMAGE_PREFIX)$(shell basename $(DIR)):$(IMAGE_TAG) $(DIR)/
$(SUDO) docker buildx build \
-o $(PUSH_MULTIARCH_TARGET) \
--platform linux/amd64,linux/arm64 \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR)):$(IMAGE_TAG) $(DIR)/

# Build Dockerfile.continuous-test
if [ -f $(DIR)/Dockerfile.continuous-test ]; then \
Expand All @@ -176,15 +190,15 @@ push-multiarch-%/$(UPTODATE):
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-continuous-test:$(IMAGE_TAG) $(DIR)/; \
fi;
# Build Dockerfile.distroless if it exists
if [ -f $(DIR)/Dockerfile.distroless ]; then \
$(SUDO) docker buildx build -f $(DIR)/Dockerfile.distroless \
-o $(PUSH_MULTIARCH_TARGET_DISTROLESS) \
# Build Dockerfile.alpine if it exists
if [ -f $(DIR)/Dockerfile.alpine ]; then \
$(SUDO) docker buildx build -f $(DIR)/Dockerfile.alpine \
-o $(PUSH_MULTIARCH_TARGET_ALPINE) \
--platform linux/amd64,linux/arm64 \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-distroless:$(IMAGE_TAG) $(DIR)/; \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-alpine:$(IMAGE_TAG) $(DIR)/; \
fi;

push-multiarch-mimir: ## Push mimir docker image.
Expand Down Expand Up @@ -740,8 +754,8 @@ integration-tests: ## Run all integration tests.
integration-tests: cmd/mimir/$(UPTODATE)
go test -tags=requires_docker,stringlabels ./integration/...

integration-tests-race: ## Run all integration tests with race-enabled Mimir-distroless docker image.
integration-tests-race: export MIMIR_IMAGE=$(IMAGE_PREFIX)mimir-distroless:$(IMAGE_TAG_RACE)
integration-tests-race: ## Run all integration tests with race-enabled distroless docker image.
integration-tests-race: export MIMIR_IMAGE=$(IMAGE_PREFIX)mimir:$(IMAGE_TAG_RACE)
integration-tests-race: cmd/mimir/$(UPTODATE_RACE)
go test -timeout 30m -tags=requires_docker,stringlabels ./integration/...

Expand Down
10 changes: 4 additions & 6 deletions cmd/mimir/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Provenance-includes-location: https://github.com/cortexproject/cortex/cmd/cortex/Dockerfile
# Provenance-includes-license: Apache-2.0
# Provenance-includes-copyright: The Cortex Authors.
# We use different base images for mimir and mimir race images since race-detector needs glibc packages.
# See difference between distroless static and base-nossl at https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md.

FROM alpine:3.19.1
ARG EXTRA_PACKAGES
RUN apk add --no-cache ca-certificates tzdata $EXTRA_PACKAGES
ARG BASEIMG=gcr.io/distroless/static-debian12
FROM ${BASEIMG}
# Expose TARGETOS and TARGETARCH variables. These are supported by Docker when using BuildKit, but must be "enabled" using ARG.
ARG TARGETOS
ARG TARGETARCH
Expand Down
10 changes: 6 additions & 4 deletions cmd/mimir/Dockerfile.distroless → cmd/mimir/Dockerfile.alpine
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# SPDX-License-Identifier: AGPL-3.0-only
# We use different base images for mimir and mimir race images since race-detector needs glibc packages.
# See difference between distroless static and base-nossl at https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md.
# Provenance-includes-location: https://github.com/cortexproject/cortex/cmd/cortex/Dockerfile
# Provenance-includes-license: Apache-2.0
# Provenance-includes-copyright: The Cortex Authors.

ARG BASEIMG=gcr.io/distroless/static-debian12
FROM ${BASEIMG}
FROM alpine:3.19.1
ARG EXTRA_PACKAGES
RUN apk add --no-cache ca-certificates tzdata $EXTRA_PACKAGES
# Expose TARGETOS and TARGETARCH variables. These are supported by Docker when using BuildKit, but must be "enabled" using ARG.
ARG TARGETOS
ARG TARGETARCH
Expand Down
Loading