Cannot find kernel image or kernel config in /boot some time after boot

[markusboehme]

Image I'm using: metal-dev from develop on x86_64

What I expected to happen: I can find vmlinuz and config in /boot at any time when entering the host via sheltie, or by inspecting the host file system via the admin container.

What actually happened: Some time after boot, the contents of the EFI System Partition are available via /boot, but the contents of the actual Bottlerocket boot partition are missing.

How to reproduce the problem: Boot. Wait. Check the contents of /boot occasionally.

[markusboehme]

From the journal I noticed that some time after boot systemd-gpt-auto-generator creates a mount unit that mounts the EFI System Partition (ESP) at /boot instead. The actual Bottlerocket boot partition that was mounted by prairiedog gets unmounted.

systemd-gpt-auto-generator uses systemd's own Boot Loader Specification to decide to mount the ESP at /boot instead, based on the ESP's GUID in the partition table. The generator can mount both a distro's classic boot partition and the ESP by treating the classic boot partition as the XBOOTLDR partition and assigning it the specified GUID. However, on Bottlerocket with its two partition banks, the XBOOTLDR partition cannot be determined solely by looking at the GUIDs--there's two boot partitions, of which only one is used at any time.

I suggest we disable systemd-gpt-auto-generator (see its docs) and leave file system mounting to the explicitly defined mount units and prairiedog. We can do so by passing systemd.gpt_auto=0 as an init option on the kernel command line.

Alternatively, we may disable ESP/XBOOTLDR mounting via a patch to systemd, but I don't think we use of any of systemd-gpt-auto-generator's functionality.