ci: Add a Containerfile-based workflow

[cgwalters]

This is a small but notable step towards making the build process more container native. The rpm-ostree bits are hidden much more. At a high level, the build process outputs a "nested container" - a container image with a /nested.ociarchive at the top level.

Higher level build processes need not be aware of exactly how that .ociarchive is constructed (as it will definitely change in the future).

In an ideal world of course we wouldn't need this "wrapped image" as it runs the ergonomics. See discussion in e.g. coreos/rpm-ostree#4688
for that.

[cgwalters]

Also to make this really work we need to modify the Konfux pipelines, which I think will best be done with a new "buildah-nested" variant of https://github.com/redhat-appstudio/build-definitions/blob/main/task/buildah/0.1/buildah.yaml that knows how to extract the /nested.ociarchive.

[jlebon]

This is pretty cool.

FWIW, I got this to work without any nesting:

FROM quay.io/centos/centos:stream9 as repos

FROM quay.io/centos-bootc/bootc-image-builder:latest as builder
ARG MANIFEST=centos-stream-9.yaml
# XXX: we should just make sure our in-tree c9s repo points to the c9s paths and doesn't require vars to avoid these steps entirely
COPY --from=repos /etc/dnf/vars /etc/dnf/vars
COPY --from=repos /etc/yum.repos.d/centos.repo c9s.repo
COPY --from=repos /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial /etc/pki/rpm-gpg
# rpm-ostree doesn't honor /etc/dnf/vars right now
RUN for n in $(ls /etc/dnf/vars); do v=$(cat /etc/dnf/vars/$n); sed -ie s,\$${n},$v, c9s.repo; done
RUN --mount=type=cache,target=/workdir --mount=type=bind,rw=true,src=.,dst=/buildcontext,bind-propagation=shared rpm-ostree compose image --cachedir=/workdir --format=ociarchive --initialize /buildcontext/${MANIFEST} /buildcontext/out.ociarchive

FROM oci-archive:./out.ociarchive
# Need to reference builder here to force ordering. But since we have to run
# something anyway, we might as well cleanup after ourselves.
RUN --mount=type=bind,from=builder,src=.,target=/var/tmp --mount=type=bind,rw=true,src=.,dst=/buildcontext,bind-propagation=shared rm /buildcontext/out.ociarchive

$ podman build --cap-add=all --device /dev/fuse --security-opt label=disable --build-arg MANIFEST=centos-tier-0-stream9.yaml -t localhost/test-container -f Containerfile
...

$ podman inspect localhost/test-container:latest | jq '.[0].Labels["ostree.bootable"]'
"true"

I think if we went this route as the way to build things, we should probably just fix up the c9s repos to simplify this even further.

[cgwalters]

Thanks so much for looking at this...it'd be really really useful if we can avoid the nested output!

But that doesn't work for me, I get:

[3/3] STEP 1/2: FROM oci-archive:./out.ociarchive
Error: creating build container: creating temp directory: archive file not found: "/out.ociarchive"

Though that's with podman on MacOS:

$ podman version
Client:       Podman Engine
Version:      4.8.2
API Version:  4.8.2
Go Version:   go1.21.5
Git Commit:   aa546902fa1a927b3d770528565627d1395b19f3
Built:        Mon Dec 11 07:44:02 2023
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.9.3
API Version:  4.9.3
Go Version:   go1.21.7
Built:        Mon Feb 19 10:40:48 2024
OS/Arch:      linux/arm64

and maybe there's something special to podman-machine/podman-remote here....ah indeed, it works for me directly on

$ podman version
Client:       Podman Engine
Version:      4.9.3
API Version:  4.9.3
Go Version:   go1.21.7
Built:        Mon Feb 19 10:41:34 2024
OS/Arch:      linux/amd64

Neat! But...ugh this clashes badly with our need for having the container in a podman-machine VM to do privileged code execution to make disk images. Hmm...maybe we need to take this to a podman issue. But, supporting this outside of a podman-machine would be extremely useful to use for our production builds. OK...I've updated the PR with your changes, let's see if the GHA can successfully build.

[jlebon]

Nice, looks like GHA built it.
LGTM overall, though looks like the commit message still references the previous wrapping approach.

[cgwalters]

LGTM overall, though looks like the commit message still references the previous wrapping approach.

Fixed, thanks!

[cgwalters]

@lmilbaum please check this out when you get a second