feat: Add supports for vDPA interface

[makoto126]

Tested with SR-IOV Network Device Plugin and modified kube-ovn(ovs-dpdk offload to our DPU).
Live migration previously tested in Kubevirt(with qemu).

VM spec

apiVersion: virt.virtink.smartx.com/v1alpha1
kind: VirtualMachine
spec:
  instance:
    interfaces:
      - name: pod
      - name: vdpa1
        vdpa: { numQueues: 9, iommu: False }
      - name: vdpa2
        vdpa: { numQueues: 9, iommu: False }
  networks:
    - name: pod
      pod: {}
    - name: vdpa1
      multus:
        networkName: offload-ovn1
    - name: vdpa2
      multus:
        networkName: offload-ovn2

SR-IOV device plugin config

apiVersion: v1
kind: ConfigMap
metadata:
  name: sriovdp-config
  namespace: kube-system
data:
  config.json: |
    {
        "resourceList": [
            {
                "resourceName": "jaguar",
                "resourcePrefix": "jaguarmicro.com",
                "selectors": {
                    "vendors": ["1f53"],
                    "devices": ["1000"],
                    "drivers": ["jaguar"],
                    "vdpaType": "vhost"
                }
            }
        ]
    }

NetworkAttachmentDefinition Spec

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: offload-ovn1
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/resourceName: jaguarmicro.com/jaguar
spec:
  config: >-
    {
        "cniVersion": "0.3.0",
        "type": "kube-ovn",
        "server_socket": "self.dpu.com:12345",
        "provider": "offload-ovn1.default.ovn"
    }
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: offload-ovn2
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/resourceName: jaguarmicro.com/jaguar
spec:
  config: >-
    {
        "cniVersion": "0.3.0",
        "type": "kube-ovn",
        "server_socket": "self.dpu.com:12345",
        "provider": "offload-ovn2.default.ovn"
    }

Pod network status

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.v1.cni.cncf.io/network-status: |-
      [{
          "name": "cbr0",
          "interface": "eth0",
          "ips": [
              "10.244.2.45"
          ],
          "mac": "5a:61:29:39:0d:29",
          "default": true,
          "dns": {},
          "gateway": [
              "10.244.2.1"
          ]
      },{
          "name": "default/offload-ovn1",
          "interface": "net1",
          "ips": [
              "10.1.0.20"
          ],
          "mac": "00:00:00:03:6C:84",
          "dns": {},
          "device-info": {
              "type": "vdpa",
              "version": "1.1.0",
              "vdpa": {
                  "parent-device": "test1",
                  "driver": "vhost",
                  "path": "/dev/vhost-vdpa-2",
                  "pci-address": "0000:68:00.5"
              }
          }
      },{
          "name": "default/offload-ovn2",
          "interface": "net2",
          "ips": [
              "10.2.0.2"
          ],
          "mac": "00:00:00:50:A4:81",
          "dns": {},
          "device-info": {
              "type": "vdpa",
              "version": "1.1.0",
              "vdpa": {
                  "parent-device": "test2",
                  "driver": "vhost",
                  "path": "/dev/vhost-vdpa-3",
                  "pci-address": "0000:68:00.6"
              }
          }
      }]

VM after dhclient

ubuntu@ubuntu-container-rootfs:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 5a:61:29:39:0d:29 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    inet 10.244.2.45/24 metric 100 brd 10.244.2.255 scope global ens4
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:00:00:03:6c:84 brd ff:ff:ff:ff:ff:ff
    altname enp0s6
    inet 10.1.0.20/24 brd 10.1.0.255 scope global dynamic ens6
       valid_lft 2305sec preferred_lft 2305sec
4: ens7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:00:00:50:a4:81 brd ff:ff:ff:ff:ff:ff
    altname enp0s7
    inet 10.2.0.2/24 brd 10.2.0.255 scope global dynamic ens7
       valid_lft 2305sec preferred_lft 2305sec

[CLAassistant]

All committers have signed the CLA.

[fengye87]

Thank you for your contribution. Please also doc this new feature in this file

[fengye87]

Better name it IOMMU to keep it in Golang style

[makoto126]

The VdpaConfig struct and other structs in pkg/cloudhypervisor/client.go used Iommu too.

type VdpaConfig struct {
	Id         string `json:"id,omitempty"`
	Iommu      bool   `json:"iommu,omitempty"`
	NumQueues  int    `json:"num_queues"`
	Path       string `json:"path"`
	PciSegment int16  `json:"pci_segment,omitempty"`
}

[fengye87]

Yes, that's generated. For non-generated code, better keep them in style

[makoto126]

@fengye87 I updated the doc and IOMMU.

[codecov]

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (30780fe) 34.11% compared to head (0f3d995) 34.08%.

❗ Current head 0f3d995 differs from pull request most recent head a49cb4e. Consider uploading reports for the commit a49cb4e to get more accurate results

Files	Patch %	Lines
pkg/controller/vm_webhook.go	50.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #98      +/-   ##
==========================================
- Coverage   34.11%   34.08%   -0.04%     
==========================================
  Files           6        6              
  Lines        1952     1957       +5     
==========================================
+ Hits          666      667       +1     
- Misses       1182     1185       +3     
- Partials      104      105       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.