[RFE] Hetzner support

[tormath1]

Current situation

There is no Flatcar images for Hetzner.

Impact

Folks can't simply use Flatcar on Hetzner.

Ideal future situation

Folks can deploy Flatcar on Hetzner without additional steps.

Implementation options

• Ignition support (>3913.0.0)
• Afterburn support
• OEM support on Flatcar (i.e build images with Hetzner OEM ID + additional configurations like starting afterburn service for example): OEM: Provide Hetzner Images scripts#1880
• Documentation: docs(hetzner): update for new OEM images flatcar-website#336
• Mantle support (test suite): [wip] platform: add Hetzner mantle#533, platform: Hetzner API implementation mantle#536
• Test automation on Flatcar: ci-automation: add hetzner testing scripts#2142

Additional information

• Build and deploy a snapshot with Packer: https://github.com/apricote/flatcar-packer-hcloud/
• Deploy with Terraform: https://www.flatcar.org/docs/latest/installing/cloud/hetzner/
• Hetzner injects root password

[LenzGr]

Just for reference, there's also https://github.com/Thor77/hetzner-flatcar

[apricote]

These PRs add basic support for Hetzner (3rd checkbox in the list above):

• afterburn: set hostname for hetzner OEM bootengine#94
• afterburn: set ssh keys for hetzner OEM init#118
• OEM: Provide Hetzner Images scripts#1880

(Disclaimer: This was done because of personal interest, not in my capacity as a Hetzner employee)

[apricote]

With the above PRs merged, to quote @tormath1 in chat:

According to the release planning, we should get Hetzner images for the Alpha and Beta in the next release. 🥳

What Works

These change already support:

• Setting the hostname properly
• Reading the SSH Key from the Hetzner Cloud metadata service and setting it for the core user
• Reading metadata variables through Afterburn and making them available to other units

Whats still missing

As far as I can tell we are still missing the following functionality for feature parity with the official images.

All of these feature are very much oriented at the beginner users, who are probably not running Flatcar. I do not think we require them, and adding a note about this in the docs should be fine.

hc-utils

units for configuring networks

This seems to work out-of-the-box on flatcar

Units for volume automounts

This is just a udev rules file, should be easy to get in the oem sysext. One issue I encountered was, that the hc-utils file modified /etc/fstab which is not an option on Flatcar. Not sure how to dynamically generate and persist systemd .mount files here

Root Passwords

Initial Root Password

On Hetzner Cloud its possible to start a server without SSH Key. We generate a root password and show it to the user once. This is then configured on the server through cloud-init chpasswd for the root user.

Root Password Reset

Its possible to reset the root password through the API. This is being handled by qemu-guest-agent on the official images.

I found that qemu-guest-agent on Flatcar was only installed on the amd64 board, not on arm64.

Another issue is, that the message received is for the root user. Not sure if we can just intercept this and always modify the core user instead, or if we want to accept that this does not work.

[tormath1]

Hetzner are images are now available on Alpha and Beta 🥳
Next steps are:

• Documentation docs(hetzner): update for new OEM images flatcar-website#336
• Mantle support (test suite) for "official support"
• Test automation on Flatcar for "official support"