-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch to using OSBuild for most CoreOS boot image building #1653
Comments
I edited your text to s/image/disk image/ for osbuild. |
We discussed this in the Community Meeting Today:
|
First step of enablement for this in the pipeline: |
OK last week we started building using OSBuild for In order to compare a little I decided to write a small helper utility to do size comparison. diffbuildsizes.py.txt Here is the output for the OSBuild built artifacts:
What I really don't understand is the sizes for the qemu image. The uncompressed size is much smaller, but the compressed size is about 185MiB larger. All of this analysis is done using the size values from the meta.json file. |
OK I think I figured this out. OSBuild does the qcow2 conversion with compression on. This means the contents are internally compressed, which explains why the "uncompressed" size is much better ( |
We'll open a PR upstream to make compression an option for this stage, but for now we'll just disable it in the code because we know we don't want it: coreos/fedora-coreos-tracker#1653 (comment)
We'll open a PR upstream to make compression an option for this stage, but for now we'll just disable it in the code because we know we don't want it: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is. Optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
Modify the stages/org.osbuild.qemu stage such that compression is optional. This resolves the image size differences between an image built with coreos assember vs osbuild, as discussed in: coreos/fedora-coreos-tracker#1653 (comment)
We previously did this in a different way (2a8d1e6) but then had to revert it (39fdd61) because it caused images to not boot [1]. The root cause appears to have been the virtiofs mount not being unmounted cleanly from the supermin VM and that is now fixed so let's switch back to not compressing since we rely on our outer compression [2]. [1] coreos#3728 [2] coreos/fedora-coreos-tracker#1653 (comment)
We previously did this in a different way (2a8d1e6) but then had to revert it (39fdd61) because it caused images to not boot [1]. The root cause appears to have been the virtiofs mount not being unmounted cleanly from the supermin VM and that is now fixed so let's switch back to not compressing since we rely on our outer compression [2]. [1] #3728 [2] coreos/fedora-coreos-tracker#1653 (comment)
ok a lot of work has happened here in the background. osbuild/osbuild#1589 was the last big piece IIUC. I've now enabled We'll enable it for |
With the switch of the Note that most other images (aws, gcp, etc) are actually derived from the qemu artifact so in a way those are also indirectly built using OSBuild too. We'll switch them over to direct building as we add the functionality to support those. |
The fix for this went into |
We missed this when we switched the |
They are now on F40 and we can roll this out! xref: coreos/fedora-coreos-tracker#1653
They are now on F40 and we can roll this out! xref: coreos/fedora-coreos-tracker#1653
When closing this ticket, let's file another ticket to track that follow-up work. That way also, others interested in this can subscribe to it. Edit: actually, filed it at #1725 |
PR to make building using OSBuild the default in COSA: coreos/coreos-assembler#3800 |
The fix for this went into |
They are now on F40 and we can roll this out! xref: coreos/fedora-coreos-tracker#1653
includes updates for the following: - 2024-05-21: - coreos/fedora-coreos-tracker#1653 - 2024-06-03 - 2024-06-17: - coreos/fedora-coreos-tracker#1722
includes updates for the following: - 2024-05-21: - coreos/fedora-coreos-tracker#1653 - 2024-06-03 - 2024-06-17: - coreos/fedora-coreos-tracker#1722
includes updates for the following: - 2024-05-21: - coreos/fedora-coreos-tracker#1653 - 2024-06-03 - 2024-06-17: - coreos/fedora-coreos-tracker#1722
includes updates for the following: - 2024-05-21: - coreos/fedora-coreos-tracker#1653 - 2024-06-03 - 2024-06-17: - coreos/fedora-coreos-tracker#1722
Since the beginning in CoreOS we have inherited or developed tooling inside COSA (CoreOS Assembler) to take the created OSTree commit and stick that into all of our different disk images that we ship (qemu, aws, openstack, ISO, etc).
Since we are the only consumers of COSA we have the desire to not maintain the bespoke image building code over the long term. The consensus is to use Image Builder/OSBuild for disk image building in the future. A good amount of work has gone into enabling OSBuild to be able to do this already.
This will also allow anyone who has created a container layer (see https://github.com/coreos/layering-examples/) to build a boot image from a derived container layer using OSBuild and not requiring COSA.
For the first round of this we will focus on building the
qemu
,metal
andmetal4k
images. Most other images that we build are actually based off of theqemu
image so they will also be indirectly affected (in the future we'll use OSBuild to build them directly).Out of scope for this first round is the ISO/PXE artifacts as those are a bit more complicated.
The text was updated successfully, but these errors were encountered: