-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gf-platformid: use qemu-img create
instead of reflinks
#935
Conversation
68c8ae2
to
4eebbce
Compare
I probably would have made a
But OK as is. This must have something to do with the pattern of writes that the guestfish qemu process issues to the backing file. Or maybe it's around doing a write-read-write cycle, if something is getting corrupted on the read. |
One tangentially related topic: We are likely to have a requirement for RHCOS at some point to ship updated bootimages inside containers. It'd be nicer if we did something like have just the |
Instead of repeating `cp -a --reflink=auto` everywhere, let's just ship a handy helper script to make this less verbose. Prep for next patch.
CI shows that you'll need to source |
4eebbce
to
ff19182
Compare
OK, I added a I ended up just shipping it in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
There seems to be a recent (likely kernel) regression with reflinks where the image spit out by `gf-platformid` has its `/boot` partition subsequently corrupted. So as a safety measure, just disable all reflinks for now. See the next patch for more details.
ff19182
to
5278063
Compare
I spent a few hours yesterday trying to figure out why my Azure images were failing to boot in the cloud with: error: ../../grub-core/fs/fshelp.c:257:file `/grub2/i386-pc/normal.mod' not found. What it came down to was whether or not we were doing a reflink copy in `gf-platformid`. (My root filesystem is configured with reflink support, so `--reflink=auto` would use reflinks.) I seem to be hitting a kernel regression of some kind where reflinks + interactions with guestfish cause the `grub2/` directory of the boot partition to become corrupted: ``` $ guestmount -a tmp/azure.vhd -m /dev/sda1 tmp/mount && ll /srv/rhcos/tmp/mount ls: cannot access 'grub2': Bad message total 20 drwxrwxr-x. 2 root root 1024 Nov 19 11:55 efi d?????????? ? ? ? ? ? grub2 -rw-rw-r--. 1 root root 0 Nov 19 11:56 ignition.firstboot lrwxrwxrwx. 1 root root 8 Nov 19 11:56 loader -> loader.1 drwxr-xr-x. 3 root root 1024 Nov 19 11:56 loader.1 drwx------. 2 root root 12288 Nov 19 11:55 lost+found drwxrwxr-x. 3 root root 1024 Nov 19 11:56 ostree ``` In fact, reproducing the cloud failure is easy to do locally too, because qemu-kvm will happily boot VHDs, so one can just do `cosa run -d /path/to/vhd`. I haven't filed a bug report for this yet because I've been trying to get a more minimal reproducer and bisecting kernel versions without much success (for one, it doesn't seem to reproduce in VMs). The previous patch disabled reflinks globally. Though this makes `gf-platformid` much slower because it has to copy the full qcow2. Another more lightweight method here is to just create a new qcow2 and use the source as its backing file. (Not to mention it also makes it much faster for all the other devs/CI pipelines which *don't* run with reflinks on).
5278063
to
d47d3e6
Compare
See also #936 |
This approach (specifically the last commit) does not work, see #999. |
kola/tests/ignition: add test for versioned s3 objects
The original PR was mistakenly merged without a MinVersion in coreos#935.
I spent a few hours yesterday trying to figure out why my Azure images
were failing to boot in the cloud with:
What it came down to was whether or not we were doing a reflink copy in
gf-platformid
. (My root filesystem is configured with reflink support,so
--reflink=auto
would use reflinks.)I seem to be hitting a kernel regression of some kind where reflinks +
interactions with guestfish cause the
grub2/
directory of the bootpartition to become corrupted:
In fact, reproducing the cloud failure is easy to do locally too,
because qemu-kvm will happily boot VHDs, so one can just do
cosa run -d /path/to/vhd
.I haven't filed a bug report for this yet because I've been trying to
get a more minimal reproducer and bisecting kernel versions without much
success (for one, it doesn't seem to reproduce in VMs).
The previous patch disabled reflinks globally. Though this makes
gf-platformid
much slower because it has to copy the full qcow2.Another more lightweight method here is to just create a new qcow2 and
use the source as its backing file. (Not to mention it also makes it
much faster for all the other devs/CI pipelines which don't run with
reflinks on).