Fedora Atomic tracking issue

[dghubble]

With the introduction of Fedora Atomic as alpha, there are several issues we'll track at a high level.

• Fedora 28
  • DigitalOcean: updated from Fedora 27 to 28 in Update Fedora Atomic from 27 to 28 on DigitalOcean #225
  • AWS: Updated from Fedora 27 to 28 in Update Fedora Atomic from 27 to 28 on AWS #258
  • Google: Updated from Fedora 27 to 28 in Update Fedora Atomic from 27 to 28 on GCP #259
  • Bare-Metal: Updated from Fedora 27 to 28 in Update Fedora Atomic from 27 to 28 on bare-metal #263
• The Ingress controller addon is known to not work on Google Cloud. Solved by Use global HTTP/TCP proxy load balancing for Ingress on GCP #252
• Switch CNI plugin location from /opt/cni to /var/cni or somewhere else
  • Plugins may be lost rpm-ostree updates. Restarting flannel / calico required
  • Without an atomic-reboot-coordinator, requires ssh'ing to nodes to run atomic host upgrade
• Fedora Atomic doesn't officially support Google Cloud. We've provided instructions to do it yourself.
• Leverage firewalld (disabled). Currently we rely on cloud firewalls just like Container Linux, but this can be improved.
• No story for coordinated drains and reboots of nodes for OS upgrades.
• On AWS, implement node self-deletion on shutdown to avoid preempted spot instances piling up and triggering alerts (we have to do this on Container Linux AWS as well). See Remove node self-deletion behavior on GCP and DO #207

Notes:

• bare-metal: Fedora Atomic can fail in dracut on bare-metal IBM systems if in-band IPMI over USB interface is enabled since it was preferred over NICs. Disabling was a work-around.
• bare-metal: I've had considerable difficulty getting Fedora Atomic to install on some IBM systems. I've mostly ended up validating clusters on Supermicro. In short, if you're going to use Fedora Atomic, evaluate it on your hardware. Or just use Container Linux which is more mature/reliable.
• bare-metal: Do not prepare the ostree repo on an older system (Fedora 25 in my case). Preparing the repo with rpm-ostree-2018.4-1.fc28.src.rpm solved the sysimage mount issue.
• AWS: Use the default network_mtu rather than setting it to 8981 to take advantage of jumbo packets. The kernel regressed MTU detection recently. Container Linux has been patched (rel: Interface MTU change on AWS instances supporting Jumbo packets coreos/bugs#2443). Fedora has not.

You should NOT have any issues with log, exec, port-forward, or any addons. Recovery from control plane reboots and total cluster power cycles works correctly. Prometheus alerts should all be green as well. If you observe otherwise, please mention.

[dghubble]

Fedora Atomic will be replaced by Fedora CoreOS, so I'm not planning to invest effort into pushing these modules beyond alpha. Their value lies in aiding the addition of Fedora CoreOS or in experimenting with ideas being developed in Atomic for inclusion in Fedora CoreOS.

[dghubble]

Fedora Atomic 29 has been released, but I haven't seen it published to Digital Ocean.