support derived declarative container builds

[cgwalters]

We should support something like:

from: quay.io/fedora/fedora-silverblue:latest

in a manifest/treefile and have everything in the treefile work on top (declarative package installs, change detection etc.).

I'd also like to generalize this to something that allows pulling artifacts from a container image, like the pattern one can do with a Dockerfile

FROM quay.io/exampleimage as artifacts
...
COPY --from=artifacts /usr/bin/foo /usr/bin/foo

One important thing to decide with this is whether the output is "re-chunked" or not. For large derived builds, rechunking is probably necessary. But it means that the layers won't be shared with the base.

[Arcitec]

Thank you Colin, this sounds absolutely brilliant (and vital) for easily making derivatives of images.

As a starting point, merely being able to "pull in the treefile of another upstream treefile" and then do our own derived modifications on that and "build it all from scratch" would be a huge improvement.

One concern though is what to do about unwanted packages that the upstream treefile contains. Let's say that upstream has "install: xyz" which is a package that has a "Recommends: widget". And our downstream treefile has a "exclude: widget", then the result should be built without widget. That part should be fairly easy to solve since it's just a matter of merging an extra exclude-instruction.

But better yet, it would be even greater if it becomes possible to remove instructions from the upstream. So if upstream had a "install gnome-terminal" and we would be able to say "no... don't do that", that would be a very good improvement. Perhaps leave such a capability for later, via a new tag, or re-purposing some existing treefile tag to handle that instruction.

A final consideration for the ability to include treefiles would be the ability to nest them. So, let's say we make a treefile that derives "from silverblue:latest". Then a downstream user wants their own thing, so they say "from our-thing:latest". The downstream should then be built using the combination of Silverblue, our thing and their thing. That would hopefully be a trivial feature if all previously mentioned treefile merging/exclude features are possible, by just doing incremental merging of each treefile's instructions when combining them all.

---

As for pulling pre-built OCI layers from an upstream image, that makes a lot of sense too, to reduce build times. But like you said, it would be impossible for it to know how to share layers / figure out OCI chunking if it isn't building the whole thing from scratch itself. It would require a lot of thought to figure out a system that makes layer importing as painless as possible.

One possible idea: It imports the upstream layers and analyzes the container layers one by one and imports their contents as-is into the final derived image. So: 1. Download upstream OCI container and its layers, 2. Analyze every layer and commit every layer into new layers, including its final "RPM database/metadata layer" too. 3. Then run our own modifications on top of that, to create further layers.

[cgwalters]

This also relates to #4505