[BUG] "config file not found" when used in net-attach-def with KubeVirt and multus

[e3b0c442]

Describe the bug
When uses as a network-attachment-definition with KubeVirt and multus, trying to launch a VM throws a config file not found error with the launcher pod stuck in ContainerCreating

Expected behavior
The launcher pod (and VM) launch successfully

To Reproduce
Steps to reproduce the behavior:

1. Add a network-attachment-definition using multus
2. Add a KubeVirt VirtualMachine with a network config using the referenced network-attachment-definition
3. Try to boot VM and virt-launcher pod stuck in ContainerCreating with FailedCreatePodSandbox events in kubectl describe pod

Environment:

• Whereabouts version : whereabouts v0.7.0 linux/amd64
• Kubernetes version (use kubectl version): Client Version: v1.29.4+k3s1 note: I have whereabouts installed in the correct place with the other k3s CNIs: /var/lib/rancher/k3s/data/current/bin
• Network-attachment-definition:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: br-multus
  namespace: vms
spec:
  config: |
    {
      "cniVersion": "1.0.0",
      "name": "br-multus",
      "type": "bridge",
      "bridge": "br-multus",
      "ipam": {
        "type": "whereabouts",
        "range": "192.0.2.0/24",
        "gateway": "192.0.2.1",
        "dns": {
          "nameservers" : ["192.0.2.1", "192.0.2.2"],
          "domain": "example.com"
        }
      }
    }

• Whereabouts configuration (on the host): N/A (I don't see anywhere in the documentation that says a host configuration is necessary when using net-attach-def)
• OS (e.g. from /etc/os-release): N/A

NAME="Fedora Linux"
VERSION="39 (Thirty Nine)"

• Kernel (e.g. uname -a): Linux node5.no.e3b0c442.dev 6.8.9-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 2 18:44:19 UTC 2024 x86_64 GNU/Linux
• Others: N/A

Additional info / context
Kubevirt manifest (abridged):

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: example
  namespace: vms
spec:
# ...stuff...
  template:
  # ...stuff...
    spec:
    # ...stuff...
      devices:
      # ...stuff...
        interfaces:
          - name: default
            bridge: {}
            macAddress: aa:aa:aa:aaaa:aa
      networks:
        - name: default
          multus:
            default: true
            networkName: vms/br-multus

kubectl describe pod event:

  Warning  FailedCreatePodSandBox  60s (x402 over 89m)  kubelet  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ab9ff4ccbc8b189e27c3742f73c9e7138f518c1d7554970a2f30be0207121a31": plugin type="multus" failed (add): [vms/virt-launcher-example-nqq4f/2f4cd204-37c0-46f1-b667-9eeb165abb94:br-multus]: error adding container to network "br-multus": config file not found

[reski-rukmantiyo]

anyone have clue how to solve this?

[koendelaat]

I think this might be related to: #465 (comment)

Please try the workaround in my comment