[RFE] Add ignition support in kops

[gabriel-samfira]

Current situation

When deploying a Kubernetes cluster using kops, flatcar needs to leverage coreos-cloudinit to complete the setup process. Currently kops does not support ignition.

Impact

We need to use coreos-cloudinit to support deployments of k8s via kops.

Ideal future situation

Ignition support is added in kops, and valid ignition userdata is generated when deploying distributions that leverage ignition.

Implementation options

It would be great if we had an extra flag in kops, or a config option that would prompt kops to generate proper ignition userdata when deploying a cluster.

Additional information

There are currently a number of PRs created against coreos-cloudinit, flatcar scripts repo and afterburn that enable kops clusters on flatcar without changes to kops.

The PRs are available at:

• Add multipart mime support and refactor coreos-cloudinit#21
• Detect multipart mime and ignore it scripts#823
• Write short form hostname coreos/afterburn#922

This fixes the issue for Flatcar, but we use a customized version of ignition that ignores non-ignition user-data and allows coreos-cloudinit to handle it later on. Distributions that do not ship the Flatcar customizations will simply err out with kops.

[gabriel-samfira]

Cross link to the kops feature request: kubernetes/kops#15428

[gabriel-samfira]

Issue that prompted this RFE: kubernetes/kops#15385

[andy-v-h]

It's possible to use cloudinit, the packet cloud image has cloud init enabled. I understand cloud-init and ignition are not feature compatiable, but it may be easier to get a cloud-init enable flatcar image running than support for ignition in an EC2 env

[gabriel-samfira]

cloud-config already works by using coreos-cloudinit. As stated, getting kops to work with flatcar already has PRs proposed. The point of adding ignition support in kops is to allow flatcar as well as other distributions that use ignition exclusively, to work with kops.

This preference was expressed here: kubernetes/kops#15385 (comment)