Platform Request: CloudStack

[olivierlemasle]

Apache CloudStack was supported by the old CoreOS Container Linux, so there's already a CloudStack support in Ignition and Afterburn, but broken since Fedora CoreOS switched from systemd-networkd to NetworkManager.

---

• Why is the platform important? Who uses it?

  Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform.

  CloudStack is a turnkey solution that includes the entire "stack" of features most organizations want with an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, a full and open native API, resource accounting, and a first-class User Interface (UI). CloudStack currently supports the most popular hypervisors: VMware, KVM, Citrix XenServer, Microsoft Hyper-V...

  Apache CloudStack powers thousands of clouds and billions of dollars in transactions across an array of organizations that include Apple, BT, INRIA, Royal Melbourne Institute of Technology (RMIT), SAP, Taiwan Mobile, Verizon, and WebMD, among others. A list of some of Apache CloudStack’s users are available at http://cloudstack.apache.org/users.html

• What is the official name of the platform? Is there a short name that's commonly used in client API implementations?

  Official name: Apache CloudStack
  Common name: cloudstack

• How can the OS retrieve instance userdata? What happens if no userdata is provided?

  Two userdata methods are available in CloudStack:

  • a metadata http service provided by the virtual router
  • configdrive (similar to openstack)

  Ignition and Afterburn already support both methods.

  • For Ignition, code exists but is marked experimental, and the provider is not documented. It tries getting the ignition file from both methods.
  • For Afterburn, two platforms exists and are documented: cloudstack-configdrive and cloudstack-metadata.

  However, the "metadata" solution is broken currently for both Ignition and Afterburn, as it relies on DHCP leases on /run/systemd/netif/leases/ to get the address of the virtual router. Since fcos uses NetworkManager instead of systemd-networkd, this has to be adapted (cf Read lease files from dhclient and NetworkManager on Azure and CloudStack afterburn#146)

• Does the platform provide a way to configure SSH keys for the instance? How can the OS retrieve them? What happens if none are provided?

  Yes, SSH keys can optionally be provided. They are available in metadata (both methods - see above); this is already implemented in Afterburn.

• How can the OS retrieve network configuration? Is DHCP sufficient, or is there some other network-accessible metadata service?

  DHCP

• In particular, how can the OS retrieve the system hostname?

  DHCP

• Does the platform require the OS to have a specific console configuration?

  No

• Is there a mechanism for the OS to report to the platform that it has successfully booted? Is the mechanism required?

  No mechanism

• Does the platform have an agent that runs inside the instance? Is it required? What does it do? What language is it implemented in, and where is the source code repository?

  No agent

• How are VM images uploaded to the platform and published to other users? Is there an API? What disk image format is expected?

  Images are uploaded using an API, from a HTTP(S) URL.
  Expected disk image format depends on the hypervisor used in the CloudStack zone:

  • KVM supports: QCOW2, RAW, VHD, VMDK
  • VMware supports: OVA
  • Hyper-V supports: VHD, VHDX
  • XenServer supports: VHD

• Are there any other platform quirks we should know about?

  Compression format xz may not be used for templates. Templates may use .gz, .bz2 or .zip

---

I've already started working on adding CloudStack support. My main questions are:

• Would Fedora CoreOS accept providing official images for CloudStack?
• Would it be possible to have fcos images for CloudStack in multiple formats (at least QCOW2 and OVA to support KVM and VMware)? Is it possible in the pipeline to have a single platform and several formats?
• The OVA image requires an OVF file like vmware-template.xml but without the OVF properties (guestinfo.ignition.config.data). Does that mean having another vmware template file?
• As Afterburn has two platforms cloudstack-configdrive and cloudstack-metadata whereas Ignition has only one platform cloudstack, how does Afterburn know which platform should be used? What do you think about having a single Afterburn platform "cloudstack" trying to get metadata from both methods (like Ignition does)?

[cgwalters]

Would Fedora CoreOS accept providing official images for CloudStack?

Offhand I'd say so yes. We should probably discuss this real-time at the IRC meeting tomorrow if you're able to attend.

[olivierlemasle]

Ok, I'll attend.

[lucab]

As Afterburn has two platforms cloudstack-configdrive and cloudstack-metadata whereas Ignition has only one platform cloudstack, how does Afterburn know which platform should be used? What do you think about having a single Afterburn platform "cloudstack" trying to get metadata from both methods (like Ignition does)?

The configdrive/metadata split comes from CL times. We used to have the same split on the openstack provider, and it proved to be quite a pain as it would require an explicit dropin/customization to enable the correct one.
I agree we can introduce an additional cloudstack provider which probes both methods sequentially, as we did for openstack on coreos/afterburn#462.

[jlebon]

  Expected disk image format depends on the hypervisor used in the CloudStack _zone_:
  
  * KVM supports: QCOW2, RAW, VHD, VMDK
  * VMware supports: OVA
  * Hyper-V supports: VHD, VHDX
  * XenServer supports: VHD

A few questions about this:

• Regardless of the zone, does networking and metadata fetching is the same? (E.g. is the config drive only used on some hypervisors and the metadata only on others? Or are both used on all of them?)
• Is there a significant difference in popularity between these zones?
• If we only supported a single format to start, what would be the preferred one?

[olivierlemasle]

• Userdata fetching (metadata http service vs configdrive) does not depend on the zone nor on the hypervisor used, but on the network used by the VM instance. A network has a network offering, which may provide some services (DHCP, DNS, Firewall, etc.). The service "UserData" may be provided by VirtualRouter (metadata http service) or by "ConfigDrive".
  In the same zone, some VMs can therefore have their userdata provided by the VirtualRouter whereas other VMs have userdata provided by ConfigDrive.

• I suppose that VMware is still the hypervisor the most used with CloudStack, followed by KVM and XenServer. I do not have statistics, that's what I guess from seeing CloudStack projects in various companies. Images provided by CloudStack community members for CoreOS Container Linux (and other OS) are available on 4 formats.

[lucab]

From yesterday meeting:

• we are thinking about how to add new platform without an explosion in the number of images at Consider more efficient ways of supporting additional cloud platforms #719. For Cloudstack, it looks like at minimum we would have to build-upload two new image: VHD and OVA.
• the first blocker in bootstraping this platform seems to be the DHCP-lease parsing logic for NM that we are missing, for both Afterburn and Ignition. As it looks like the metadata endpoint (when existing) is always located on the virtual router / default gw, we could maybe avoid DHCP-parsing at all and just get the default gw (e.g. ip route show default).