Uploading to cloud platforms: Packet

[dustymabe]

This is part of #146 and tracks the work/discussion around uploading to Packet.

[vielmetti]

One note to drop in re PXE, specifically iPXE - we are tracking this issue ipxe/ipxe#90 which yields intermittent failures with iPXE as a result of a bug in OCSP certificate validation. Any eyes on this especially for test and validation cycles would be helpful.

[vielmetti]

See also #135

[bgilbert]

I expect our approach will look similar to CL:

• we'll publish signed OS images for Packet, and
• Packet will mirror those to their provisioning servers, check the signatures, and run coreos-installer at provisioning time.

[vielmetti]

As noted on the 7/24/2019 IRC meeting channel, two current blocking issues are #24 (network management) and #105 (live IPXE issues).

[bgilbert]

FCOS works on Packet today using the custom_ipxe image type and the following iPXE script:

#!ipxe

set version 31.20200310.2.0
set base-url https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/${version}/x86_64
kernel ${base-url}/fedora-coreos-${version}-live-kernel-x86_64 ip=dhcp rd.neednet=1 initrd=fedora-coreos-${version}-live-initramfs.x86_64.img console=ttyS1,115200n8 coreos.inst.install_dev=/dev/sda coreos.inst.stream=testing coreos.inst.ignition_url=<URL-TO-IGNITION-CONFIG> coreos.inst.platform_id=packet
initrd ${base-url}/fedora-coreos-${version}-live-initramfs.x86_64.img
boot

Note that the above installs current FCOS testing to disk and reboots into it. It should also be possible to drop the install kargs and run directly from RAM.

Because we don't have support for Packet-specific networking, the machine only gets a public IPv4 address, but it does install and apply the Ignition config and SSH keys.

Because the install flow involves coreos-installer, and the installer supports overriding the platform ID, it may not even make sense to publish a distinct image for Packet. The Packet folks might have an opinion on that, though.

[vielmetti]

@bgilbert -

What is the brief state of #24 (network management) ? It should be possible to configure the host from the metadata service we provide, which is sufficiently similar to the AWS EC2 setup that it should be only a little bit of code to implement. That would give you access to information about the private IPv4 address and also IPv6 networking.

I would love to see an image packaged in such a way that someone could pick FCOS from a Packet menu and be up and running without any more install work than necessary. Looks like we are close to that with the iPXE script.

[bgilbert]

We're working on switching initramfs networking to NetworkManager (#394). Afterburn knows how to talk to the Packet metadata service, but it only knows how to write systemd-networkd configs and would need to be updated for NM. Alternatively, it might make sense to teach NM how to talk to the metadata service directly.

sufficiently similar to the AWS EC2 setup that it should be only a little bit of code to implement.

Hmm, any pointers for that? On EC2 we just DHCP, I believe.

[vielmetti]

@bgilbert - ah, I was commenting more that the delta between the Packet code to talk to metadata and the EC2 to talk to metadata would be small; did not realize when I wrote this that NM did not speak to the metadata at all.

[bgilbert]

The bare-metal image currently configures a serial console on ttyS0, but Packet needs it on ttyS1. We could work around this with https://github.com/coreos/coreos-installer/issues/13 but it'd be better to automatically switch kernel arguments by platform.

[jlebon]

We could work around this with coreos/coreos-installer#13 but it'd be better to automatically switch kernel arguments by platform.

How do you imagine this working? Would the installer automatically add it based on whether the specified Ignition platform ID was packet?

[bgilbert]

xref #110 (comment) for platform-specific serial console kargs.