Hypervisor Implementation Detail

[krisnova]

We need to decide what approach we want to take to create and manage virtual machines from the daemon.

https://github.com/aurae-runtime/api/pull/1 Calls out the following interface for managing VMs.

  rpc RegisterVirtualMachine(RegisterVirtualMachineRequest) returns (RegisterVirtualMachineResponse) {}

  rpc StartVirtualMachine(StartVirtualMachineRequest) returns (StartVirtualMachineResponse) {}

  rpc StopVirtualMachine(StopVirtualMachineRequest) returns (StopVirtualMachineResponse) {}

  rpc DestroyVirtualMachine(DestroyVirtualMachineRequest) returns (DestroyVirtualMachineResponse) {}

Which can be simplified to the following 4 pieces of functionality for virtual machines.

• Register()
• Start()
• Stop()
• Destroy()

[krisnova]

Firecracker could be leveraged as it could be compiled directly into the program. We would, however, need to identify how to leverage the provided jailer utility.

Example process tree could look like this:

init(1) -> jailer(2) -> auraed(3)

[MalteJ]

Currently it looks like this: One Firecracker process executes one MicroVM. One Jailer contains one Firecracker.

When we run auraed as pid 1 and auraed has support for running containers anyways, we could move the Jailer functionality into auraed (pid 1) and execute a restricted auraed process for every MicroVM, which will act as the MicroVM's hypervisor.

auraed(1) -> auraed(2) as Hypervisor ==virtualization==> MicroVM Kernel -> auraed(VM pid 1)

[krisnova]

Another thought is to leverage something like QEMU directly, or just write our own KVM abstracting in rust.

• https://docs.rs/kvm-ioctls/latest/kvm_ioctls/

[krisnova]

Right now the pid 1 demo code leverages

virtqemud

We should form an opinion on which hypervisor technology we want to leverage and we should use auraed to manage the client VMs in the future :)

[MalteJ]

libvirt sort of does the same like auraed: Managing multiple VMs. So there's some redundancy here.
Maybe we should directly manage qemu-system-x86_64 processes from auraed. Via the QMP API we can modify running VMs (attaching, disattaching devices, sending ACPI events etc.).

Qemu supports virtualized PCI busses attached to VMs. This is a requirement for SR-IOV. We need this, when we want to offload network functionality to the NIC. Or if we want to attach NVMe drives/namespaces to the VM directly, without consuming too much hypervisor CPU cycles. So I'd say we need Qemu VMs.

Further we want to support MicroVMs as a sandboxing mechanism for pods. I know of two hypervisors providing MicroVMs: Qemu MicroVMs and Firecracker.
So which one should we choose? We already got Qemu on the system. On the other hand Firecracker is written in Rust and we could directly integrate firecracker methods into auraed. As Firecracker is also under Apache License v2 we have no license issue, doing so.