-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
container-encapsulate
should be more intelligent
#4012
Comments
I think I mentioned this somewhere else before, but for solution 1, one possible heuristic might be to group packages by the SRPM field since binary packages from the same SRPM will definitely change together. |
That happens today; note the rpm-ostree/rust/src/container.rs Line 290 in 72334b9
|
I think it'd be useful to look at optimizing other containers too: |
Some work on this in #4271 |
One other thing we could consider is grouping some noarch packages together in bins. Theoretically the noarch packages are the same across all architectures so this could possibly even represent some space savings in the container registry because a noarch RPM layer between x86_64/aarch64/ppc64le/s390x would (or could) all be the same. In reality the details of how the layers are created may prevent them from being bit for bit compatible so it might not work out. Just a suggestion. Thanks for working on this. |
Yeah, one can think of this like a multi-dimensional space across which we're trying to find a local maximum. The dimensions are:
There definitely will be some sharing in basically all cases because of linux-firmware, which is so large that it always ends up in its own layer. And due to the way the rpm-ostree container builds work, that will always be bit-for-bit identical and hence shared. |
Today, the algorithm backing
rpm-ostree compose container-encapsulate
(andrpm-ostree compose image
) is pretty simplistic. It basically tries to stick the largest RPMs in layers, and then "spills" the rest to a final layer.The status quo works okay-ish for Fedora CoreOS:
There's 412 total packages in Fedora CoreOS. That second layer of 141.3MB is the "long tail" of 336 packages - meaning anytime any one of those packages change, we will re-ship that whole layer.
But it gets very much notably worse for Fedora Silverblue today (demo over here) which has 1225 packages (three times that of FCOS), occupying a total of 1.748GB (about 2.5x FCOS).
The "remainder" set is 1200 packages in 461.8MB. It seems likely that at least one of those 1200 packages changes relatively quickly - and if it does, there's a major "size amplification" here.
Solution 0: Accept externally generated data for chunking
Here we take external data using historical build information and the external process decides on the chunking.
Solution 1: Better heuristics and/or human-written suggestions
First we can just better assign packages to "layers" based on heuristics (change frequency) possibly augmented with human-written defaults. (e.g. "group ostree and rpm-ostree").
Solution 2: Create state based on prior builds
Teach the build process to accept a previous build as input (for
compose image
, we already do) and better compute which packages should go where based on the delta between them?Solution 3: Fix container runtimes to handle e.g. 500 layers
I don't think this would be really hard, but the problem is there's going to be a long tail of people running old container engines for a while and we want our images to be compatible.
(This is the same thing that will stall container zstd work)
Ultimate solution
What we really want is container image deltas of course.
The text was updated successfully, but these errors were encountered: