download linuxkit instead of building, as it is a basic tool; eliminate eve-build-<user> image from most activities

[deitch]

This changes the Makefile so that linuxkit is downloaded directly from its GitHub release page as a compiled binary, rather than downloading the repo and compiling it. It also eliminates the need for a "linuxkit.version" flag file, but checking the actual linuxkit version against the desired version by running linuxkit version --short. As of v1.3.0, linuxkit supports that syntax.

The real benefit, though, is that it eliminates the need for building that eve-build-<user> docker image, unless you take an activity that requires it (like make shell). Those activities are all developer-specific ones. With this change, it will build that only if you take one of those activities.

make eve-pillar is not one of those activities, nor is any of the other make eve-* ones for packages, or even make eve.

This will make getting to the actual build part of make commands faster.

Also, perhaps even more importantly, it eliminates the usage of that image from the GitHub Actions pipelines. That means no more pulls of golang or similar for that purpose. This will make logs cleaner, and a lot fewer pulls of rate-limited images. We were pulling them with every single run of any command to the Makefile. That goes away.

[giggsoff]

I can see tons of lines in workflow (something like tar: Ignoring unknown extended header keyword...) can you please check?

[deitch]

A few things:

First, I do not understand why the riscv64 PR build failed.

Second, it looks like make eve still tries to create eve-build-ubuntu. See for example this line (arm64, xen, generic) or this line (amd64, kvm, generic).

I cannot recreate it locally. If I try to run the same make command as in the workflow logs, I do not get it trying to build the eve-build-* image.

It does not build for the pkgs, which is a definite step up. But we should resolve the rest before merging this in.

[deitch]

I can see tons of lines in workflow (something like tar: Ignoring unknown extended header keyword...) can you please check?

Yes, I know what it is. linuxkit tracks information on each file that was placed into its tar (rootfs.tar) so it can know efficiently if it needs to redo them when rebuilding. It turns out to make it significantly faster (multiple times) to regenerate. Added in this PR. This was an efficiency idea from @rouming, which turned out to be really helpful.

It stores the additional info as PAX headers on the files. Not every version of tar recognizes extended PAX headers. Those that do not, usually treat it as a warning, which is convenient. It continues to work backwards-compatibly.

The messages you are seeing is because we call tar xf rootfs.tar in make sbom, and since gnu tar on the runner does not support those headers, it will give the message.

We probably can (should?) fix it, although unrelated to this PR. We can suppress it with tar --warning=no-unknown-keyword. Separate PR for that, methinks.

[deitch]

Mind, tar --warning= only works on GNU tar, whereas bsdtar has no issues with those. Probably need to check tar version first.

[deitch]

It looks like the packages job in build.yml is getting the latest Makefile but the eve job is not? I added a temporary commit that dumps the Makefile to see.

[deitch]

First, I do not understand why the riscv64 PR build failed.

This was transient, rerunning it fixed it.

Second, it looks like make eve still tries to create eve-build-ubuntu. See for example this line (arm64, xen, generic) or this line (amd64, kvm, generic).

This was due to an update necessary in build.yml unrelated to this PR. Yash is fixing that one, when it is done, I will rebase this PR on that fix.

[deitch]

Yash fixed it, rebased, should be clean now. Will follow closely.

[deitch]

I can confirm that the build now does not try to build linuxkit or the eve-build-ubuntu image! I checked multiple make calls in the action run logs.