-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hook up blueprint filesystem customizations #124
Conversation
One tricky part here is that we have rules in osbuild/images about what partitions a user can (not) add to avoid making unbootable images. Unfortunately those aren't exposed (yet) so we can't apply them here. Maybe we can only allow cutomizing the size of |
It feels weird to essentially be partially re-implenting things here (and will be totally confusing to users) so I think exposing more of osbuild/images is the right way to go. |
PR to move the policies into the public pkg - osbuild/images#395 |
NOTE: CI tests will fail until images is updated |
This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days. |
b8bd52f
to
2e5dc6e
Compare
2e5dc6e
to
06214f5
Compare
I've lost track of the status of this, so if you want it don't wait on me :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks pretty small and sane to me, I think we also just need to update the docs, right? (Can be done post-merge)
Hm, testing farm was failing with
|
2d65062
to
82547de
Compare
82547de
to
07d3375
Compare
I've rebased this on git main, just fixing up the (textual, but not logical) conflict with the new "double rootfs size default") code. Not tested locally beyond building, but let's see what CI says! |
Once we move to the
Please let me know if I'm missing something here or thinking too complicated. |
I feel like we keep re-treading this, not sure how to solve the logical disconnect. I'll try to write some more about this in the bootc docs. Now for sure I would like to support better things that just "unmanaged regular files" in But for broad/generic use cases, we simply have to support machine-local state - and for maximum compatibility we can't just require everything to learn to do configmaps or sysexts or something that doesn't boil down to supporting For example Anaconda today is writing an
The configuration definitely needs to be in the deployment root, yes. Also BTW let's not get too hung up on |
[..]
Thank you! Sorry that we keep coming back to this, but hopefully this was the disconnect I had. In my (wrong) mental model I assumed that there is
It seems that ostree is smart enough to know if files in (1) get changed and tracks them as local state. I hope I got this right now and sorry that it took me a while. |
Yes, https://containers.github.io/bootc/filesystem.html#etc ➡️ https://ostreedev.github.io/ostree/atomic-upgrades/#assembling-a-new-deployment-directory (I'll expand this more over there) ostree's semantics here are more arguably more "package system" like (vs "simple image system" as is emphasized by e.g. systemd's push for empty /etc) - and this definitely has tradeoffs, but again in a nutshell we support dropping arbitrary files there and they persist. |
07d3375
to
e26533f
Compare
I rebased this again...I think it still makes sense. (Sorry about being initially semi-against...I had just really hoped we also started with the container-based configuration but that is just messier than I thought) |
Once we move to |
This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days. |
This also allows the root filesystem size to be increased if the default of 10GiB isn't enough.
Make sure / is larger than the default of 10GiB and make sure there is a new /var/log mountpoint.
This requires a new release of images that exposes the policies package.
Users cannot create a mountpoint on /ostree, make sure that an error is returned when this happens.
I rebased this again (and slightly tweaked tiny bits). Unfortunately it needs more work because:
I attach the generated osbuild manifest with the exiting code: The issue is that we run "org.osbuild.bootc.install-to-filesystem" with all the mounts, i.e. "/", "/boot" but also "/var/log" now.
That should be enough (but is a bit of extra work). |
This commit tweaks the test setup slightly to use pytest.raises for exception checking and also run test_mount_ostree_error() only for the centos image (as the error checking/policies are exactly the same for both images).
The "images" library does not support custom mount points for bootc based images just yet. The reason is that images will generate an osbuild manifest that contains all the "mounts" for the generated disk. This means that with an extra partition like `/var/log` this is visible for the "bootc install-to-filesystem" stage. And that will trip up bootc because it validates the content of the target directory. Example error with `/var/log` as a custom mount point: ``` ... Installing image: docker://quay.io/centos-bootc/centos-bootc:stream9 ERROR Installing to filesystem: Verifying empty rootfs: Non-empty root filesystem; found "var" Traceback (most recent call last): File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 53, in <module> r = main(args["options"], args["inputs"], args["paths"]) File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 48, in main subprocess.run(pargs, env=env, check=True) File "/usr/lib64/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['bootc', 'install', 'to-filesystem', '--source-imgref', 'containers-storage:[overlay@/run/osbuild/containers/storage+/run/containers/storage]3b612dd1fae2437c00ae3187d0e63daa7a94711560fb1712389edd4121668c96', '--skip-fetch-check', '--generic-image', '--karg', 'rw', '--karg', 'console=tty0', '--karg', 'console=ttyS0', '--karg', 'systemd.journald.forward_to_console=1', '--target-imgref', 'quay.io/centos-bootc/centos-bootc:stream9', '/run/osbuild/mounts']' returned non-zero exit status 1. ``` So AFAICT "images" need sto be changed so that: 1. The "install-to-filesystem" stage only takes the "essential" mounts (/, /boot/, /boot/efi) 2. After "install-to-filesystem" ran we need a "org.osbuild.mkdir" stage for the extra mount points that also only mounts the "essential" mounts As a first step on the journy this commit limits customizations to "/" and "/boot" which is already very useful as many people have asked for precisely those.
As suggested by @ochosi I tweaked the PR now so that we initially only allow to customize "/" and "/boot" which will already cover many use-cases. Then we can continue on this once we have updated images to support what bib needs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fwiw, this looks good to me now but please count me as only a a +0.5 because I tweaked/rebased this code a bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, let's enable /
and /boot
and enable more later.
Really simple change :) The testing feels a bit klunky, suggestions welcome. I thought about checking the manifest, but it boots the image anyway so checking the output of
df
seemed to be the most straight forward way.