chore(ci): fix storage use, go version and lint job

[mloiseleur]

Description

There are currently multiple issues in the CI system:

1. Go version used for build All Images is still go 1.19, it was missed in Upgrade ExternalDNS to go 1.20 #3673
2. pull-external-dns-lint is launched on each PR and cannot succeed
3. Build all images is taking too much space on disk and fails on many PRs

This PR fixes this.

[mloiseleur]

/test pull-external-dns-lint

[k8s-ci-robot]

@mloiseleur: The following commands are available to trigger required jobs:

• /test pull-external-dns-lint
• /test pull-external-dns-unit-test

Use /test all to run all jobs.

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mloiseleur]

@Raffo @szuecs @johngmyers In this PR, I found a way to fix:

1. pull-external-dns-lint: by forcing download of golangci-lint within the Makefuile
   • I noticed that @johngmyers opened this PR, but I don't if / when it will be merged.
2. disk usage of Build all images: by cleaning disk between each steps
   • On a GoLang project, I am unsure if it's relevant to build those three arch on each PRs.
3. go version, which was different from the others Job

Also, I'm not sure to understand why there is two lint and if we should remove one.

[johngmyers]

The prow jobs have the advantage of retesting against the result of a proposed merge. They can also be re-run by people who aren't approvers. GitHub Actions have the advantage of being able to run in the author's fork prior to opening a PR.

The downloading of golangci-lint in the Makefile seems reasonable. I was thinking of copying another project's script, but I don't think that makes much difference.

I'm not so sure about the build-all-images change.

[johngmyers]

Is there any particular reason we need an alpine base image? Can we switch to ko and chainguard?

[johngmyers]

Okay, the build.image/multiarch target iterates over each of the architectures anyway, so that change seems fairly innocuous.
/ok-to-test
/lgtm
/assign @njuettner

[johngmyers]

Opened a WIP PR to switch the build system to ko in #3835

[mloiseleur]

@johngmyers Interesting, thanks. ko build seems far better.

[Raffo]

Is there any particular reason we need an alpine base image?

@johngmyers there's #3396 open to address that. The reason is mostly historical: distroless didn't exist when this project was created and kubernetes didn't have kubectl debug and thus alpine was used, to have something with a package manager (although we run as non root). I think it is reasonable now to change to something else. I would recommend google's distroless as it is what other base images are using.

[mloiseleur]

@Raffo @johngmyers In order to unblock current CI, I removed build on armv7. At least, FTM, all tests can pass.

I would be happy to move ko.build once #3835 is ready !

[Raffo]

This is good to go to make builds work, we can then change things with ko or whatever in another PR.

[Raffo]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mloiseleur, Raffo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Raffo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment