bib: use container based buildroot when building bootc disk images (HMS-3318) #138
Conversation
mvo5
changed the title
|
Fwiw, I tested that centos-9 can be build/booted with this branch: the "regular" version seems to be working now as well, it seems we got some updates here (where can I see those? is there some RSS, mailing list or simialr?), I will prepare updated tests in a followup (or here if that is prefered). [edit: this is done now automatically in https://github.com//pull/148] |
|
Tested locally too and on macOS and it's all looking good. Nice work! Some things I see in the build log that we might want to look into (perhaps in a follow-up).
Seems to be happening on pretty much every file:
|
|
Hmm is the osbuild stage trying to overwrite the labels for the ostree state, i.e. the stuff in |
|
Thanks Colin and Achilleas (I'm probably stating the obvious) - the issue seems to be that we run our org.osbuild.selinux stage and we set the file_contexts there: when I do this: |
👍 |
We shouldn't be trying to run That said, to fix this type of stuff we currently need to ensure that |
|
Noticed something while looking into the issues here (and #149) (should've been caught in the pipeline PR in images). |
|
I spent some time trying to debug and understand this today. One thing I am struggling with is I don't yet have an efficient edit-compile-debug-cycle with trying to patch Go code in osbuild/images, then rebuild this project and rerun. And I don't understand osbuild debugging either (the lack of anything like I do want to level set again that our existing base images now have (since this landed) a single one (ok two) liner to create a correct sparse raw disk image by executing the container itself. No copying the image around (3? at least?) times, SELinux works correctly, we honor the bootc install config and kargs etc. (There's no support for injecting external user data or partitions, but that's currently somewhat intentional)
(hmm, I guess we should add the default image size into the container image or its metadata, then we could add This invocation uses the container as the buildroot (like this PR tries to), so it fixes the biggest problem that bib can't target c9s (and a cascade of related problems, like forcing everyone to fetch RPMs just to build the disk image for their container and understanding cache management for those distinct from their container images). Now of course this raw disk image is just a base thing that still needs significant logic to handle e.g. converting to a format for uploading to AWS, etc. and it doesn't make sense (IMO) to ship that in the OS itself, but in a tool like this. One thing I briefly tried to dig into a bit more then is how to chain these two into one invocation, which gets into two related problems:
Now related to the second, I actually don't see how #120 can work today without doing something similar to what bootc does for running skopeo from the host mountns, or just trying to bind mount in the container storage (I'd need to ask about best practices for that). |
Hmm yes...probably from the ostree stuff ( |
This commit adds a new `exclude` option to the container-deploy stage. This is needed when we deploy `bootc` containers that are used for the buildroot. Here the `/sysroot` dir needs to be excluded because it has conflicting selinux definitions for files there and in the normal "root" dir. See also osbuild/bootc-image-builder#138
This commit adds a new `exclude` option to the container-deploy stage. This is needed when we deploy `bootc` containers that are used for the buildroot. Here the `/sysroot` dir needs to be excluded because it has conflicting selinux definitions for files there and in the normal "root" dir. See also osbuild/bootc-image-builder#138
This commit adds a new `exclude` option to the container-deploy stage. This is needed when we deploy `bootc` containers that are used for the buildroot. Here the `/sysroot` dir needs to be excluded because it has conflicting selinux definitions for files there and in the normal "root" dir. See also osbuild/bootc-image-builder#138
|
Fwiw, this should be unblocked by: Once merged we just need to update the go.mod and the selinux issue should be resolved. [edit: those are merged now and both fedora and centos stream are build and boot, see https://github.com//pull/148] |
This commit adds a new `exclude` option to the container-deploy stage. This is needed when we deploy `bootc` containers that are used for the buildroot. Here the `/sysroot` dir needs to be excluded because it has conflicting selinux definitions for files there and in the normal "root" dir. See also osbuild/bootc-image-builder#138
mvo5
force-pushed
the
buildroot-from-container3
branch
from
f25439c
to
156a764
Compare
mvo5
force-pushed
the
buildroot-from-container3
branch
from
156a764
to
8cc046c
Compare
|
Looks like the TMT failure is on selinux denials
That could actually be new in this PR? |
|
One thing I guess I should say is that I recently noticed that bootc's attempt to detect |
|
OK I did containers/bootc#284 and yeah now I'm just in a battle to try to gain install_t again...I would swear this code worked at some point. I need to trace through whether what's happening here is working and if so how it differs. |
|
I would like to eliminate |
Thank you, yes, that is a regression. I think i know what is wrong and opened osbuild/images#402 [edit: this has landed and things are green everywhere except macOS where go dies during the container build, investigating] |
Instead of using the trandition rpm based buildroot this commit switches the buildroot to the target bootc container. The rational is that the low-level tooling like mkfs etc should match the target system.
mvo5
force-pushed
the
buildroot-from-container3
branch
from
8cc046c
to
4db2d1c
Compare
This unbreak macos tests and also fedora-bootc:eln got fixed/reverted upstream.
mvo5
force-pushed
the
buildroot-from-container3
branch
from
0800fbd
to
f0eea67
Compare
Distro-specific runners try to set up a build environment to satisfy
certain expectations for stages (things they might expect from a live OS
environment). One of the things the Fedora runner does is run
`systemd-tmpfiles --create`. This causes a lot of warnings in our new
buildroot, like the following:
/usr/lib/tmpfiles.d/rpm-ostree-autovar.conf:31: Failed to resolve user 'rpcuser': No such process
likely because we are running in an "ostree-based" tree that isn't fully
deployed (/etc/passwd only contains root).
The Linux runner is a fallback runner that doesn't perform any
preparations. This should be adequate for the stages we use in bib.
achilleas-k
force-pushed
the
buildroot-from-container3
branch
from
6055684
to
f02cfee
Compare
|
so not a caching issue 🙄 |
achilleas-k
force-pushed
the
buildroot-from-container3
branch
from
f02cfee
to
f9b3fbe
Compare
|
golangci-lint is failing for unknown reasons. Let's force merge this after everything else succeeds and see if it also happens on the merge queue after rebase. |
|
Fwiw, the changes from @achilleas-k look good to me (thank you for your help here!) |
I dropped the linter requirement from the merge rules instead. I'll enable it again later. |
Now that osbuild/bootc-image-builder#138 is fixed.
Instead of using the trandition rpm based buildroot this commit
switches the buildroot to the target bootc container. The rational
is that the low-level tooling like mkfs etc should match the target
system.
Once that has landed and we can use container based buildroots
I will start working on
bootc install-filesysteminstead of the ostreepipeline.