Switch to multistage build Dockerfiles for VPA

[voelzmo]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Switch to multistage Dockerfiles to build the multi-platform binaries with docker buildx. The previous setup with multiple make targets was quite complicated and isn't necessary to achieve what we're trying to do. In particular, this change does:

• remove some make targets, as they are no longer needed: build-binary-with-vendor, build-binary-with-vendor-*, sub-push, docker-builder, build-in-docker, build-in-docker-*
• remove the second docker-build execution from the docker-push make target, so we only build the image once
• Move the logic from build-in-docker to a build stage in the Dockerfile for each component
  • Making use of the built-in variables for desired build platform, like TARGETARCH

Special notes for your reviewer:

The interface for someone cutting a new release of the VPA stays the same: the command to be executed is still make release, as documented.

I used to following process to verify that the images built from master and with this change are identical:

• If you're going to reproduce this yourself, for all the following steps, make sure to replace REGISTRY with a container registry where you have access to :)
• On current master, go to vertical-pod-autoscaler/pkg/recommender and execute

REGISTRY=voelzmo ALL_ARCHITECTURES="amd64 arm64" TAG="from-master" make release

• checkout PR branch (rebase on current master, if necessary) and execute

REGISTRY=voelzmo ALL_ARCHITECTURES="amd64 arm64" TAG="from-multistage-build" make release

• Verify there are no differences in the files on the two images by comparing the two images using container-diff with

container-diff diff daemon://voelzmo/vpa-recommender-amd64:from-master daemon://voelzmo/vpa-recommender-amd64:from-multistage-build --type=file

• to be sure, you can repeat the last step for all component-platform combinations

Does this PR introduce a user-facing change?

NONE

[kwiesmueller]

/assign @sophieliu15

[sophieliu15]

I am pretty overloaded with oncall issues. @laoj2 Could you take a look at this PR?

[laoj2]

here and below: can we use the latest Go 1.20 (1.20.12 as of today https://go.dev/doc/devel/release) to reduce the chances of vulnerabilities in the next release?

[voelzmo]

Oh, you're right, I forgot to push the latest commit that bumps the builder image, such that it is on the same version as the currently used builder – sorry!
In the light of the currently open PR by dependabot, I updated it to go 1.21.5. If you follow the steps outlined above to verify that there's no difference between the images, make sure that both images are built with the same go version.

[laoj2]

Thanks!

/lgtm

@kwiesmueller can you take a look?

[k8s-ci-robot]

@laoj2: changing LGTM is restricted to collaborators

In response to this:

Thanks!

/lgtm

@kwiesmueller can you take a look?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kwiesmueller]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kwiesmueller, voelzmo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• vertical-pod-autoscaler/OWNERS [kwiesmueller]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment