Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use usespace vdpa can't find the pfName because the net path is wrong #507

Closed
JasinlikeEatingOrange opened this issue Nov 8, 2023 · 20 comments · May be fixed by #514
Closed

use usespace vdpa can't find the pfName because the net path is wrong #507

JasinlikeEatingOrange opened this issue Nov 8, 2023 · 20 comments · May be fixed by #514

Comments

@JasinlikeEatingOrange
Copy link

What happened?

There is no net path in Virtio-net is the net layer of virtio, and the standard kernel driver places this layer separately into the virtio1 path
e74c25da4e1dc57466fd8e40174e5812

What did you expect to happen?

If the net folder is not found in the /sys/bus/pci/devices/pciAddr/physfn/net path, I hope you can find the folder starting with virtio again

What are the minimal steps needed to reproduce the bug?

No**/sys/bus/pci/devices/pciAddr/physfn/net**
Only the /sys/bus/pci/devices/pciAddr virtioxx/physfn/net

Anything else we need to know?

0fcfcfcf1d5fd87ff5f527946522b39c
a80b6274adbf5a1d9f766357260cebd0
5bed714b280cbf48fc35600bc7e7532d

Component Versions

Please fill in the below table with the version numbers of components used.

Component Version
SR-IOV Network Device Plugin 0bf8f4f
SR-IOV CNI Plugin 0bf8f4f
Multus
Kubernetes v1.24.0
OS ubuntu 22.04

Config Files

Config file locations may be config dependent.

Device pool config file location (Try '/etc/pcidp/config.json')
Multus config (Try '/etc/cni/multus/net.d')
CNI config (Try '/etc/cni/net.d/')
Kubernetes deployment type ( Bare Metal, Kubeadm etc.)
Kubeconfig file
SR-IOV Network Custom Resource Definition

Logs

SR-IOV Network Device Plugin Logs (use kubectl logs $PODNAME)

root@master:~/temp/kubevirt# kubectl logs -f -n kube-system kube-sriov-device-plugin-amd64-d6vsq
I1108 14:43:33.442520 1 manager.go:51] Using Kubelet Plugin Registry Mode
I1108 14:43:33.442648 1 main.go:44] resource manager reading configs
I1108 14:43:33.442685 1 manager.go:79] raw ResourceList: {
"resourceList": [{
"resourceName": "sriov_dpu",
"resourcePrefix": "yusur.tech",
"selectors": {
"rootDevices": ["0000:01:00.0#1-6"]
}
}
]
}
I1108 14:43:33.442759 1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[] RootDevices:[0000:01:00.0#1-6] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1108 14:43:33.442781 1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc0001b32b0}]
I1108 14:43:33.442815 1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1108 14:43:33.442818 1 main.go:60] Discovering host devices
I1108 14:43:33.469296 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02 unknown unknown
I1108 14:43:33.469535 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02 unknown unknown
I1108 14:43:33.469605 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02 unknown unknown
I1108 14:43:33.469659 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02 unknown unknown
I1108 14:43:33.469719 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02 unknown unknown
I1108 14:43:33.469767 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02 unknown unknown
I1108 14:43:33.469814 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02 unknown unknown
I1108 14:43:33.469875 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02 unknown unknown
I1108 14:43:33.469931 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02 unknown unknown
I1108 14:43:33.469990 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470009 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470035 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470045 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470054 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470064 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470075 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470086 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470097 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470106 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470127 1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02 Red Hat, Inc. Virtio network device
I1108 14:43:33.470151 1 main.go:66] Initializing resource servers
I1108 14:43:33.470157 1 manager.go:105] number of config: 1
I1108 14:43:33.470160 1 manager.go:109]
I1108 14:43:33.470162 1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1108 14:43:33.470166 1 manager.go:111] DeviceType: netDevice
I1108 14:43:33.470339 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.470511 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.470694 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.471309 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.471644 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.471959 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.472276 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1108 14:43:33.472473 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1108 14:43:33.472583 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1108 14:43:33.472592 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1108 14:43:33.472827 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.472837 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1108 14:43:33.472977 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.472984 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1108 14:43:33.473145 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.473154 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1108 14:43:33.473338 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.473347 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1108 14:43:33.473572 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.473580 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1108 14:43:33.473825 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.473834 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1108 14:43:33.474096 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1108 14:43:33.474104 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1108 14:43:33.474171 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1108 14:43:33.474178 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1108 14:43:33.474239 1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1108 14:43:33.474246 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1108 14:43:33.474261 1 factory.go:106] device added: [pciAddr: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474263 1 factory.go:106] device added: [pciAddr: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474265 1 factory.go:106] device added: [pciAddr: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474267 1 factory.go:106] device added: [pciAddr: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474271 1 factory.go:106] device added: [pciAddr: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474273 1 factory.go:106] device added: [pciAddr: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1108 14:43:33.474281 1 manager.go:139] New resource server is created for sriov_dpu ResourcePool
I1108 14:43:33.474285 1 main.go:72] Starting all servers...
I1108 14:43:33.474446 1 server.go:199] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1108 14:43:33.474818 1 server.go:226] sriov_dpu device plugin endpoint started serving
I1108 14:43:33.474842 1 main.go:77] All servers started.
I1108 14:43:33.474848 1 main.go:78] Listening for term signals
I1108 14:43:33.964954 1 server.go:110] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1108 14:43:33.964962 1 server.go:134] ListAndWatch(sriov_dpu) invoked
I1108 14:43:33.964969 1 server.go:142] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},},}

Multus logs (If enabled. Try '/var/log/multus.log' )
Kubelet logs (journalctl -u kubelet)
Closed
@adrianchiris
Copy link
Contributor

I dont understand the issue,

device plugin is configured using rootDevices selector and is reporting 6 devices under ysur.tech/sriov_dpu
it emits warning in logs because it did not find netdevice for the PFs which is expected in your case since the PF driver did not create a netdevice.

CNI is failing since it failed to find netdevice.

@JasinlikeEatingOrange
Copy link
Author

I dont understand the issue,

device plugin is configured using rootDevices selector and is reporting 6 devices under ysur.tech/sriov_dpu it emits warning in logs because it did not find netdevice for the PFs which is expected in your case since the PF driver did not create a netdevice.

CNI is failing since it failed to find netdevice.

Sorry, let me rephrase.My current problem is that pod can’t find pfname

1. Situation

Now the directory of our virtio device on this machine is as follows.
/sys/bus/pci/devices/0000:01:00.5/physfn/virtio5/net

The current version of sriov reads the pfname of this machine as shown below.
dd56c5f2f7d65efd489893a7dd968a86
it such as /sys/bus/pci/devices/0000:01:00.5/physfn/net
It cannot find the net directory of this device, so it cannot be placed in the resource pool.

2. Solution

I added an extra logical judgment to the path here, and now it can be successfully adapted to our hardware equipment. We are a Chinese DPU equipment company YUSUR. Could you please evaluate whether it’s necessary to integrate this into the community?
image

@adrianchiris
Copy link
Contributor

adrianchiris commented Nov 16, 2023 •
edited

PF name the the PF netdev name of the PCI device.
what you suggest is to support in device plugin to take the name of the PF netdev is from a virtio device

saying in the topic the path is "wrong" sort of hints that device plugin behaves incorrectly, which is not the case.

I still dont understand what it is that you are seeking. currently device plugin will indeed fail to get PF name, but would still expose the resources.

if you want to make this virtio netdev available in workload pod then you need support in CNI not device plugin.

can you join one of our community meetings and present your use-case ? this would help

@JasinlikeEatingOrange
Copy link
Author

Certainly, I can attend the meeting. Meanwhile, please review my reproduction on your end. Please refrain from referring to my previous responses; focus on the replies I provide below.

1.Current original version:

#Local situation, where the node node1vdpa is a DPU node. I want to use our SR-IOV plugin to obtain the corresponding VF resources.
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl  get pods -A -owide
NAMESPACE        NAME                                   READY   STATUS    RESTARTS       AGE     IP               NODE        NOMINATED NODE   READINESS GATES
kube-system      coredns-bcb458c48-kxlbh                1/1     Running   0              7m10s   10.244.1.3       master      <none>           <none>
kube-system      coredns-bcb458c48-qwqfn                1/1     Running   0              7m10s   10.244.1.4       master      <none>           <none>
kube-system      etcd-master                            1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-apiserver-master                  1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-controller-manager-master         1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-cd7nn                   1/1     Running   0              7m9s    192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-nntrh                   1/1     Running   0              7m9s    192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-scheduler-master                  1/1     Running   22 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-5p72l   1/1     Running   0              27s     192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-vgd9j   1/1     Running   0              7m9s    192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-db-77b8bbd66f-ccwpk            2/2     Running   0              11m     192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-master-78b7865c59-xntjb        2/2     Running   0              10m     192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-4vs7s                     3/3     Running   0              9m12s   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-dpu-host-vdpa-cv7m2       1/1     Running   0              8m11s   192.168.201.21   node1vdpa   <none>           <none>
ovn-kubernetes   ovs-node-2w6qr                         1/1     Running   0              12m     192.168.201.20   master      <none>           <none>

Let's first take a look at the log information for the SR-IOV pod.

#SR-IOV log information

root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-5p72l
I1123 02:59:05.191954       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1123 02:59:05.192234       1 main.go:44] resource manager reading configs
I1123 02:59:05.192262       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
           "pfNames": ["enp1s0#1-6"]
        }
      }
  ]
}
I1123 02:59:05.192348       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[enp1s0#1-6] RootDevices:[] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1123 02:59:05.192370       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc000243110}]
I1123 02:59:05.192408       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1123 02:59:05.192412       1 main.go:60] Discovering host devices
I1123 02:59:05.269156       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 02:59:05.275501       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 02:59:05.275589       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 02:59:05.275653       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 02:59:05.275702       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 02:59:05.275752       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 02:59:05.275799       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 02:59:05.275864       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 02:59:05.275920       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 02:59:05.275984       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276007       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276029       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276042       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276052       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276061       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276072       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276082       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276093       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276210       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276239       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276291       1 main.go:66] Initializing resource servers
I1123 02:59:05.276435       1 manager.go:105] number of config: 1
I1123 02:59:05.276441       1 manager.go:109]
I1123 02:59:05.276444       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1123 02:59:05.276446       1 manager.go:111] DeviceType: netDevice
I1123 02:59:05.276764       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.276982       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277159       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277376       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277596       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277821       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.278080       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.278169       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1123 02:59:05.278261       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1123 02:59:05.278273       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1123 02:59:05.278479       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278489       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1123 02:59:05.278618       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278627       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1123 02:59:05.278768       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278777       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1123 02:59:05.278941       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278952       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1123 02:59:05.279140       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279149       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1123 02:59:05.279351       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279359       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1123 02:59:05.279603       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279612       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1123 02:59:05.279670       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1123 02:59:05.279679       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1123 02:59:05.279738       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1123 02:59:05.279747       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1123 02:59:05.279755       1 manager.go:125] no devices in device pool, skipping creating resource server for sriov_dpu
I1123 02:59:05.279758       1 main.go:72] Starting all servers...
I1123 02:59:05.279760       1 main.go:77] All servers started.
I1123 02:59:05.279763       1 main.go:78] Listening for term signals

it will throw error "unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
Below is the node resource information. We can see that the node has not obtained SR-IOV resource information: yusur.tech/sriov_dpu: 0

#Node Resource Information
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl describe node node1vdpa
Name:               node1vdpa
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    k8s.ovn.org/dpu-host=
                    k8s.ovn.org/vdpa-dpu-host=
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=node1vdpa
                    kubernetes.io/os=linux
                    node=node1
                    yusur-type=dpu
Annotations:        k8s.ovn.org/host-addresses: ["172.17.0.1","192.168.124.1","192.168.200.35","192.168.201.101"]
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"brp0_node1vdpa","mac-address":"00:15:6d:ee:ce:cb","ip-addresses":["192.168.201.21/24"],"ip-add...
                    k8s.ovn.org/node-chassis-id: 58674eff-bed3-4cec-837d-afc88df23dc4
                    k8s.ovn.org/node-gateway-router-lrp-ifaddr: {"ipv4":"100.64.0.3/16"}
                    k8s.ovn.org/node-mgmt-port-mac-address: 0a:58:0a:f4:00:02
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"192.168.201.21/24"}
                    k8s.ovn.org/node-subnets: {"default":"10.244.0.0/24"}
                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/containerd/containerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Tue, 21 Nov 2023 13:51:38 +0800
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  node1vdpa
  AcquireTime:     <unset>
  RenewTime:       Thu, 23 Nov 2023 11:04:08 +0800
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 23 Nov 2023 10:59:16 +0800   Thu, 23 Nov 2023 10:51:27 +0800   KubeletReady                 kubelet is posting ready status. AppArmor enabled
Addresses:
  InternalIP:  192.168.201.21
  Hostname:    node1vdpa
Capacity:
  cpu:                   8
  ephemeral-storage:     24590672Ki
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                8127040Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
Allocatable:
  cpu:                   8
  ephemeral-storage:     22662763278
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                6795840Ki
  pods:                  110
  yusur.tech/sriov_dpu:  0
System Info:
  Machine ID:                 eabba479306945cfb24292224ef23d1a
  System UUID:                da7512b0-6152-49be-9bf6-de13d986c2c7
  Boot ID:                    a2a0a5dc-308e-4db6-a362-8c3d7a855a3a
  Kernel Version:             5.15.0-87-generic
  OS Image:                   Ubuntu 20.04.6 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.19
  Kubelet Version:            v1.24.2
  Kube-Proxy Version:         v1.24.2
PodCIDR:                      10.244.1.0/24
PodCIDRs:                     10.244.1.0/24
Non-terminated Pods:          (3 in total)
  Namespace                   Name                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                    ------------  ----------  ---------------  -------------  ---
  kube-system                 kube-multus-ds-nntrh                    100m (1%)     100m (1%)   50Mi (0%)        50Mi (0%)      11m
  kube-system                 kube-sriov-device-plugin-amd64-5p72l    250m (3%)     1 (12%)     40Mi (0%)        200Mi (3%)     5m7s
  ovn-kubernetes              ovnkube-node-dpu-host-vdpa-cv7m2        100m (1%)     0 (0%)      300Mi (4%)       0 (0%)         12m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource              Requests     Limits
  --------              --------     ------
  cpu                   450m (5%)    1100m (13%)
  memory                390Mi (5%)   250Mi (3%)
  ephemeral-storage     0 (0%)       0 (0%)
  hugepages-1Gi         0 (0%)       0 (0%)
  hugepages-2Mi         300Mi (25%)  300Mi (25%)
  yusur.tech/sriov_dpu  0            0
Events:
  Type    Reason        Age                From     Message
  ----    ------        ----               ----     -------
  Normal  NodeNotReady  27m (x2 over 15h)  kubelet  Node node1vdpa status is now: NodeNotReady
  Normal  NodeReady     12m (x4 over 45h)  kubelet  Node node1vdpa status is now: NodeReady

In other words, before modifying the code version, our SR-IOV cannot obtain node information in the current situation when using selectors for pfNames. At the same time, it will throw an error, "unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory". To understand the error situation, let's go to the node node1vdpa and investigate what's going on.

2. Node information for node1vdpa

Let's first take a look at the top-level directory. Currently, we are on the node1vdpa node.

root@node1vdpa:/sys/bus/pci/devices# ls
0000:00:00.0  0000:00:01.2  0000:00:01.5  0000:00:02.0  0000:00:02.3  0000:00:02.6  0000:00:0a.0  0000:00:0a.3  0000:00:0a.6  0000:00:0c.0  0000:00:1f.2  0000:01:00.1  0000:01:00.4  0000:01:00.7  0000:0b:00.0  0000:0e:00.0
0000:00:01.0  0000:00:01.3  0000:00:01.6  0000:00:02.1  0000:00:02.4  0000:00:02.7  0000:00:0a.1  0000:00:0a.4  0000:00:0a.7  0000:00:1b.0  0000:00:1f.3  0000:01:00.2  0000:01:00.5  0000:02:00.0  0000:0c:00.0  0000:10:00.0
0000:00:01.1  0000:00:01.4  0000:00:01.7  0000:00:02.2  0000:00:02.5  0000:00:03.0  0000:00:0a.2  0000:00:0a.5  0000:00:0b.0  0000:00:1f.0  0000:01:00.0  0000:01:00.3  0000:01:00.6  0000:09:00.0  0000:0d:00.0


root@node1vdpa:/sys/bus/pci/devices# cd  /sys/bus/pci/devices/0000:01:00.3/

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3# ls
ari_enabled           config                    current_link_width  dma_mask_bits    enable       irq            local_cpus      modalias  numa_node  power_state   resource   resource4_wc         subsystem         uevent
broken_parity_status  consistent_dma_mask_bits  d3cold_allowed      driver           iommu        link           max_link_speed  msi_bus   physfn     reset         resource1  revision             subsystem_device  vendor
class                 current_link_speed        device              driver_override  iommu_group  local_cpulist  max_link_width  msi_irqs  power      reset_method  resource4  sriov_vf_msix_count  subsystem_vendor  virtio10

After entering the directory /sys/bus/pci/devices/0000:01:00.3, we can indeed see that the net folder is not present. Let's take a look at how the code is implemented in the project to understand this.
image

We can see that the current code navigates into the physfn directory within /sys/bus/pci/devices/0000:01:00.3 to look for the net folder.

#let's see physfn dir
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3# cd physfn

# we cant find the net dir meanwhile we can find virtio1
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ls
ari_enabled           consistent_dma_mask_bits  device           enable         irq            max_link_speed  msi_irqs     remove        resource      revision                 sriov_offset     sriov_vf_total_msix  uevent   virtfn2  virtfn6
broken_parity_status  current_link_speed        dma_mask_bits    firmware_node  link           max_link_width  numa_node    rescan        resource1     rom                      sriov_stride     subsystem            vendor   virtfn3  virtio1
class                 current_link_width        driver           iommu          local_cpulist  modalias        power        reset         resource4     sriov_drivers_autoprobe  sriov_totalvfs   subsystem_device     virtfn0  virtfn4
config                d3cold_allowed            driver_override  iommu_group    local_cpus     msi_bus         power_state  reset_method  resource4_wc  sriov_numvfs             sriov_vf_device  subsystem_vendor     virtfn1  virtfn5

It's evident that the net folder is not present in the current directory, so it cannot find the corresponding resources. As a result, the local SR-IOV resources are reported as 0.

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ethtool -i enp1s0
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

3. Solution

When the driver is virtio, we can observe that, in the directory /sys/bus/pci/devices/0000:01:00.3/physfn, an additional folder named virtioxx is present. Let's enter that folder.

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ls
ari_enabled           consistent_dma_mask_bits  device           enable         irq            max_link_speed  msi_irqs     remove        resource      revision                 sriov_offset     sriov_vf_total_msix  uevent   virtfn2  virtfn6
broken_parity_status  current_link_speed        dma_mask_bits    firmware_node  link           max_link_width  numa_node    rescan        resource1     rom                      sriov_stride     subsystem            vendor   virtfn3  virtio1
class                 current_link_width        driver           iommu          local_cpulist  modalias        power        reset         resource4     sriov_drivers_autoprobe  sriov_totalvfs   subsystem_device     virtfn0  virtfn4
config                d3cold_allowed            driver_override  iommu_group    local_cpus     msi_bus         power_state  reset_method  resource4_wc  sriov_numvfs             sriov_vf_device  subsystem_vendor     virtfn1  virtfn5

# virtio1 is our focus
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# cd virtio1/

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn/virtio1# ls
device  driver  features  modalias  net  power  status  subsystem  uevent  vendor

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.6/physfn/virtio1# ls net/
enp1s0

Great, we can now see the net folder with enp1s0 inside, which is the pfName we need. So, the code can be modified as follows:
image

4. The effect after the modification.

#I have already switched to the image with the modified code on node1vdpa. Now, let's delete the existing pod 
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl delete po -n kube-system      kube-sriov-device-plugin-amd64-5p72l --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "kube-sriov-device-plugin-amd64-5p72l" force deleted

#get pods
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl get pods -A -owide
NAMESPACE        NAME                                   READY   STATUS    RESTARTS       AGE   IP               NODE        NOMINATED NODE   READINESS GATES
kube-system      coredns-bcb458c48-kxlbh                1/1     Running   0              45m   10.244.1.3       master      <none>           <none>
kube-system      coredns-bcb458c48-qwqfn                1/1     Running   0              45m   10.244.1.4       master      <none>           <none>
kube-system      etcd-master                            1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-apiserver-master                  1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-controller-manager-master         1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-cd7nn                   1/1     Running   0              45m   192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-nntrh                   1/1     Running   0              45m   192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-scheduler-master                  1/1     Running   22 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-8g52f   1/1     Running   0              3s    192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-vgd9j   1/1     Running   0              45m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-db-77b8bbd66f-ccwpk            2/2     Running   0              49m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-master-78b7865c59-xntjb        2/2     Running   0              48m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-4vs7s                     3/3     Running   0              47m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-dpu-host-vdpa-cv7m2       1/1     Running   0              46m   192.168.201.21   node1vdpa   <none>           <none>
ovn-kubernetes   ovs-node-2w6qr                         1/1     Running   0              50m   192.168.201.20   master      <none>           <none>

#check the pod logs.
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-8g52f
I1123 03:37:46.852262       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1123 03:37:46.852408       1 main.go:46] resource manager reading configs
I1123 03:37:46.852435       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
           "pfNames": ["enp1s0#1-6"]
        }
      }
  ]
}
I1123 03:37:46.852517       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0002a0e00]
I1123 03:37:46.852526       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc00041a840 AdditionalInfo:map[] SelectorObjs:[0xc0002a0e00]}]
I1123 03:37:46.852554       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1123 03:37:46.852562       1 main.go:62] Discovering host devices
I1123 03:37:46.891928       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 03:37:46.892463       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 03:37:46.892618       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 03:37:46.892702       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 03:37:46.892775       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 03:37:46.892840       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 03:37:46.892903       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 03:37:46.892966       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 03:37:46.893028       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 03:37:46.893111       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 03:37:46.893135       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893164       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893176       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893186       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893197       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893208       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893221       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893234       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893248       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893276       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893304       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1123 03:37:46.893321       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1123 03:37:46.893324       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1123 03:37:46.893327       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1123 03:37:46.893330       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1123 03:37:46.893335       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1123 03:37:46.893338       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1123 03:37:46.893341       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1123 03:37:46.893346       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1123 03:37:46.893349       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1123 03:37:46.893352       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893357       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893363       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893367       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893371       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893373       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893378       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893381       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893384       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893392       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893395       1 main.go:68] Initializing resource servers
I1123 03:37:46.893402       1 manager.go:117] number of config: 1
I1123 03:37:46.893407       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1123 03:37:46.893410       1 manager.go:122] DeviceType: netDevice
I1123 03:37:46.893505       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.893752       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.893982       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894218       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894488       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894771       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.895066       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.895460       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.895659       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.895879       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896124       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896395       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896670       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896956       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.897259       1 manager.go:138] initServers(): selector index 0 will register 6 devices
I1123 03:37:46.897269       1 factory.go:122] device added: [identifier: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897272       1 factory.go:122] device added: [identifier: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897275       1 factory.go:122] device added: [identifier: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897277       1 factory.go:122] device added: [identifier: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897280       1 factory.go:122] device added: [identifier: 0000:01:00.6, vendor: 1af4, device: 1041, driver: vfio-pci]
I1123 03:37:46.897284       1 factory.go:122] device added: [identifier: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897303       1 manager.go:156] New resource server is created for sriov_dpu ResourcePool
I1123 03:37:46.897309       1 main.go:74] Starting all servers...
I1123 03:37:46.897475       1 server.go:254] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1123 03:37:46.898073       1 server.go:282] sriov_dpu device plugin endpoint started serving
I1123 03:37:46.898109       1 main.go:79] All servers started.
I1123 03:37:46.898115       1 main.go:80] Listening for term signals
I1123 03:37:47.847008       1 server.go:116] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1123 03:37:47.847085       1 server.go:157] ListAndWatch(sriov_dpu) invoked
I1123 03:37:47.847107       1 server.go:170] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},},}

Let's check the SR-IOV resources on the local node.

root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl describe node node1vdpa
Name:               node1vdpa
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    k8s.ovn.org/dpu-host=
                    k8s.ovn.org/vdpa-dpu-host=
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=node1vdpa
                    kubernetes.io/os=linux
                    node=node1
                    yusur-type=dpu
Annotations:        k8s.ovn.org/host-addresses: ["172.17.0.1","192.168.124.1","192.168.200.35","192.168.201.101"]
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"brp0_node1vdpa","mac-address":"00:15:6d:ee:ce:cb","ip-addresses":["192.168.201.21/24"],"ip-add...
                    k8s.ovn.org/node-chassis-id: 58674eff-bed3-4cec-837d-afc88df23dc4
                    k8s.ovn.org/node-gateway-router-lrp-ifaddr: {"ipv4":"100.64.0.3/16"}
                    k8s.ovn.org/node-mgmt-port-mac-address: 0a:58:0a:f4:00:02
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"192.168.201.21/24"}
                    k8s.ovn.org/node-subnets: {"default":"10.244.0.0/24"}
                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/containerd/containerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Tue, 21 Nov 2023 13:51:38 +0800
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  node1vdpa
  AcquireTime:     <unset>
  RenewTime:       Thu, 23 Nov 2023 11:39:37 +0800
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 23 Nov 2023 11:37:55 +0800   Thu, 23 Nov 2023 10:51:27 +0800   KubeletReady                 kubelet is posting ready status. AppArmor enabled
Addresses:
  InternalIP:  192.168.201.21
  Hostname:    node1vdpa
Capacity:
  cpu:                   8
  ephemeral-storage:     24590672Ki
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                8127040Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
Allocatable:
  cpu:                   8
  ephemeral-storage:     22662763278
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                6795840Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
System Info:
  Machine ID:                 eabba479306945cfb24292224ef23d1a
  System UUID:                da7512b0-6152-49be-9bf6-de13d986c2c7
  Boot ID:                    a2a0a5dc-308e-4db6-a362-8c3d7a855a3a
  Kernel Version:             5.15.0-87-generic
  OS Image:                   Ubuntu 20.04.6 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.19
  Kubelet Version:            v1.24.2
  Kube-Proxy Version:         v1.24.2
PodCIDR:                      10.244.1.0/24
PodCIDRs:                     10.244.1.0/24
Non-terminated Pods:          (3 in total)
  Namespace                   Name                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                    ------------  ----------  ---------------  -------------  ---
  kube-system                 kube-multus-ds-nntrh                    100m (1%)     100m (1%)   50Mi (0%)        50Mi (0%)      47m
  kube-system                 kube-sriov-device-plugin-amd64-8g52f    250m (3%)     1 (12%)     40Mi (0%)        200Mi (3%)     2m
  ovn-kubernetes              ovnkube-node-dpu-host-vdpa-cv7m2        100m (1%)     0 (0%)      300Mi (4%)       0 (0%)         48m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource              Requests     Limits
  --------              --------     ------
  cpu                   450m (5%)    1100m (13%)
  memory                390Mi (5%)   250Mi (3%)
  ephemeral-storage     0 (0%)       0 (0%)
  hugepages-1Gi         0 (0%)       0 (0%)
  hugepages-2Mi         300Mi (25%)  300Mi (25%)
  yusur.tech/sriov_dpu  0            0
Events:
  Type    Reason     Age                From     Message
  ----    ------     ----               ----     -------
  Normal  NodeReady  48m (x4 over 45h)  kubelet  Node node1vdpa status is now: NodeReady

JasinlikeEatingOrange added a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 29, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
3957a58 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
Open
JasinlikeEatingOrange added a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
958b9e3 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange added a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
5fac342 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange added a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
1bc6105 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange pushed a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
Fix can't get pfname for virtio pci path
2e68d1e 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange pushed a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
Fix can't get pfname for virtio pci path
c220256 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange pushed a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
c5050b4 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
JasinlikeEatingOrange pushed a commit to JasinlikeEatingOrange/sriov-network-device-plugin that referenced this issue Nov 30, 2023
@JasinlikeEatingOrange
Fix can't get pfname for virtio pci path
6615dfc 
Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address,
preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources.

Fixes k8snetworkplumbingwg#507

Signed-off-by: JasinlikeEatingOrange <jasin_smith@outlook.com>
@SchSeba
Copy link
Collaborator

SchSeba commented Dec 5, 2023

Hi,
please be aware that pfName is not for regular interfaces. you need to use the pciAddress if you want to add the devices to the sriov-device-plugin.

if not you will have an issue if you have a pod running using the device and you restart the device plugin he will not be able to find the device after the reboot.

@adrianchiris
Copy link
Contributor

adrianchiris commented Dec 5, 2023 •
edited

@JasinlikeEatingOrange can you change pfName selector to rootDevices selector and use PCI address of the physical device ?
this selector also support VF index range syntax

@SchSeba pciAddress selector will match on the exact device, i believe he needs rootDevices to get all VF devices of a given physical device.

@SchSeba
Copy link
Collaborator

SchSeba commented Dec 5, 2023

but they are not VFs if I didn't miss anything from the logs you can see

I1123 02:59:05.269156       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 02:59:05.275501       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 02:59:05.275589       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 02:59:05.275653       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 02:59:05.275702       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 02:59:05.275752       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 02:59:05.275799       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 02:59:05.275864       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 02:59:05.275920       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 02:59:05.275984       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276007       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276029       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276042       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276052       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276061       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276072       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276082       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276093       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276210       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276239       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device

they are all Red Hat, Inc. Virtio network device these is not a PF they are just virtual devices.
I can also see Capacity: cpu: 8 so I am pretty sure it's a virtual server.

@SchSeba
Copy link
Collaborator

SchSeba commented Dec 5, 2023 •
edited

these are the only selectors supported for devices that are not sriov PFs

"pciAddresses" N Target device's pci address as string string list Default: null "pciAddresses": ["0000:03:02.0"]
"acpiIndexes" N Target device's acpi index as string string list Default: null "acpiIndexes": ["101"]

@adrianchiris
Copy link
Contributor

but they are not VFs if I didn't miss anything from the logs you can see

from previous comment #507 (comment)

see section 2.

the pci device has a physfn link in sysfs and the physical device has virtfn files in sysfs, so was assuming its sriov.
maybe its not the case IDK...

@SchSeba
Copy link
Collaborator

SchSeba commented Dec 6, 2023

I don't understand also.

@JasinlikeEatingOrange can you please explain what is this device as it doesn't look like a real PF

@JasinlikeEatingOrange
Copy link
Author

I don't understand also.

@JasinlikeEatingOrange can you please explain what is this device as it doesn't look like a real PF

@SchSeba @adrianchiris
I apologize for not explaining this earlier. As a DPU manufacturer, Our actual DPU hardware devices are relatively expensive, with each DPU physical card costing approximately tens of thousands of dollars. Therefore, for development and testing purposes, we use several virtual machines to simulate these devices. When it comes to final online validation, we use physical cards. However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.
I answered your previous question. Does my response align with the inquiry from before? I'm concerned that I may have misunderstood it.

@adrianchiris
Copy link
Contributor

does rootDevices selector OR pciAddresses work for your virtual env ?

what DPU are we talking about ? company model etc ?

@SchSeba
Copy link
Collaborator

SchSeba commented Dec 7, 2023

@JasinlikeEatingOrange base on

However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.

Does this mean that the PF for your card is going to be a virtio device?

@JasinlikeEatingOrange
Copy link
Author

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

I1208 12:24:37.632987       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1208 12:24:37.647684       1 main.go:44] resource manager reading configs
I1208 12:24:37.647717       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "rootDevices": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1208 12:24:37.647806       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[] RootDevices:[0000:01:00.0#1-6] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1208 12:24:37.647831       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc0001c3110}]
I1208 12:24:37.647874       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1208 12:24:37.647879       1 main.go:60] Discovering host devices
I1208 12:24:37.810009       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1208 12:24:37.810358       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1208 12:24:37.810450       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1208 12:24:37.816184       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1208 12:24:37.816267       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1208 12:24:37.816325       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1208 12:24:37.816377       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1208 12:24:37.816440       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1208 12:24:37.816500       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1208 12:24:37.816576       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816594       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816615       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816625       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816636       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816647       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816657       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816667       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816679       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816689       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816717       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816740       1 main.go:66] Initializing resource servers
I1208 12:24:37.816748       1 manager.go:105] number of config: 1
I1208 12:24:37.816751       1 manager.go:109]
I1208 12:24:37.816753       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1208 12:24:37.816756       1 manager.go:111] DeviceType: netDevice
I1208 12:24:37.816968       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817177       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817377       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817597       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817839       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818106       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818391       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818496       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1208 12:24:37.818586       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1208 12:24:37.818595       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1208 12:24:37.818833       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.818844       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1208 12:24:37.818989       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.818997       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1208 12:24:37.819162       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819171       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1208 12:24:37.819360       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819370       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1208 12:24:37.819637       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819646       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1208 12:24:37.819888       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819897       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1208 12:24:37.820160       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.820169       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1208 12:24:37.820232       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1208 12:24:37.820242       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1208 12:24:37.820306       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1208 12:24:37.820314       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1208 12:24:37.820330       1 factory.go:106] device added: [pciAddr: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820333       1 factory.go:106] device added: [pciAddr: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820336       1 factory.go:106] device added: [pciAddr: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820338       1 factory.go:106] device added: [pciAddr: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820340       1 factory.go:106] device added: [pciAddr: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820343       1 factory.go:106] device added: [pciAddr: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820352       1 manager.go:139] New resource server is created for sriov_dpu ResourcePool
I1208 12:24:37.820356       1 main.go:72] Starting all servers...
I1208 12:24:37.820537       1 server.go:199] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1208 12:24:37.829701       1 server.go:226] sriov_dpu device plugin endpoint started serving
I1208 12:24:37.829722       1 main.go:77] All servers started.
I1208 12:24:37.829726       1 main.go:78] Listening for term signals
I1208 12:24:38.293372       1 server.go:110] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1208 12:24:38.293426       1 server.go:134] ListAndWatch(sriov_dpu) invoked
I1208 12:24:38.293431       1 server.go:142] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},},}

@JasinlikeEatingOrange
Copy link
Author

pciAddresses
Use pciAddresses to select vf resource.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-8rg72
I1208 12:31:14.488334       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1208 12:31:14.488497       1 main.go:44] resource manager reading configs
I1208 12:31:14.488519       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "pciAddresses": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1208 12:31:14.488632       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[0000:01:00.0#1-6]} PfNames:[] RootDevices:[] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1208 12:31:14.488645       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00011ad20 SelectorObj:0xc0001dd2b0}]
I1208 12:31:14.488678       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1208 12:31:14.488682       1 main.go:60] Discovering host devices
I1208 12:31:14.631963       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1208 12:31:14.632309       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1208 12:31:14.632412       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1208 12:31:14.632484       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1208 12:31:14.632558       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1208 12:31:14.632621       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1208 12:31:14.632692       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1208 12:31:14.632764       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1208 12:31:14.632827       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1208 12:31:14.632906       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.689364       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693363       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693388       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693399       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693408       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693652       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693673       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693687       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693700       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693928       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693966       1 main.go:66] Initializing resource servers
I1208 12:31:14.693974       1 manager.go:105] number of config: 1
I1208 12:31:14.693977       1 manager.go:109]
I1208 12:31:14.693980       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1208 12:31:14.693982       1 manager.go:111] DeviceType: netDevice
I1208 12:31:14.700548       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.700829       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701085       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701397       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701727       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.702054       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.702368       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.705249       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1208 12:31:14.705367       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1208 12:31:14.705382       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1208 12:31:14.705721       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.705737       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1208 12:31:14.705913       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.705928       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1208 12:31:14.706125       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706141       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1208 12:31:14.706353       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706370       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1208 12:31:14.706613       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706627       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1208 12:31:14.706932       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706946       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1208 12:31:14.707229       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.707242       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1208 12:31:14.707318       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1208 12:31:14.707332       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1208 12:31:14.707410       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1208 12:31:14.707423       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1208 12:31:14.707431       1 manager.go:125] no devices in device pool, skipping creating resource server for sriov_dpu
I1208 12:31:14.707435       1 main.go:72] Starting all servers...
I1208 12:31:14.707438       1 main.go:77] All servers started.
I1208 12:31:14.707443       1 main.go:78] Listening for term signals

@JasinlikeEatingOrange
Copy link
Author

@JasinlikeEatingOrange base on

However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.

Does this mean that the PF for your card is going to be a virtio device?

Yes, Your understanding is correct.
During the verification phase, this card is simulated by a virtual machine. When it's actually sold to customers, the displayed effects are consistent. So, you can simply understand the PF as a real physical device. There's no need to worry whether it's a real physical device or not. In fact, a real physical device would also have this error.

@adrianchiris
Copy link
Contributor

adrianchiris commented Dec 10, 2023 •
edited

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

yes, its a warning [1] as expected. it would still expose related VFs as resources.

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

Line 52 in 30652a2

glog.Warningf("unable to get PF name %q", err.Error())
****

@JasinlikeEatingOrange
Copy link
Author

JasinlikeEatingOrange commented Dec 10, 2023 •
edited

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

yes, its a warning [1] as expected. it would still expose related VFs as resources.

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

Line 52 in 30652a2

glog.Warningf("unable to get PF name %q", err.Error())

yes if i use this modify image it will work, use pfname and rootDevices works. not throw warning.
should this modify merge to repo?

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

contributor can determine if it's necessary to modify it.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-qgjzq
I1210 10:01:34.904510       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1210 10:01:34.904700       1 main.go:46] resource manager reading configs
I1210 10:01:34.904730       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "rootDevices": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1210 10:01:34.904827       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0001b8e00]
I1210 10:01:34.904839       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc000332870 AdditionalInfo:map[] SelectorObjs:[0xc0001b8e00]}]
I1210 10:01:34.904868       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1210 10:01:34.904875       1 main.go:62] Discovering host devices
I1210 10:01:35.040963       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1210 10:01:35.041013       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1210 10:01:35.041017       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1210 10:01:35.041020       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1210 10:01:35.041022       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1210 10:01:35.041025       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1210 10:01:35.041028       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1210 10:01:35.041030       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1210 10:01:35.041032       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1210 10:01:35.041035       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1210 10:01:35.041038       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041040       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041042       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041044       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041047       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041049       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041052       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041057       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041060       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041063       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041068       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1210 10:01:35.041529       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1210 10:01:35.041653       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1210 10:01:35.041787       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1210 10:01:35.041891       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1210 10:01:35.041956       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1210 10:01:35.042016       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1210 10:01:35.042088       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1210 10:01:35.042144       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1210 10:01:35.042223       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1210 10:01:35.042247       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042271       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042290       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042300       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042310       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042322       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042334       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042346       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042356       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042379       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042408       1 main.go:68] Initializing resource servers
I1210 10:01:35.042412       1 manager.go:117] number of config: 1
I1210 10:01:35.042416       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1210 10:01:35.042419       1 manager.go:122] DeviceType: netDevice
I1210 10:01:35.042498       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.042719       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.042938       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043168       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043421       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043721       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.044029       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.044448       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.044640       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.044848       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.045078       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.045330       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.053897       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.054290       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.054732       1 manager.go:138] initServers(): selector index 0 will register 6 devices
I1210 10:01:35.054741       1 factory.go:122] device added: [identifier: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054745       1 factory.go:122] device added: [identifier: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054747       1 factory.go:122] device added: [identifier: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054750       1 factory.go:122] device added: [identifier: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054752       1 factory.go:122] device added: [identifier: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054757       1 factory.go:122] device added: [identifier: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054796       1 manager.go:156] New resource server is created for sriov_dpu ResourcePool
I1210 10:01:35.054803       1 main.go:74] Starting all servers...
I1210 10:01:35.055023       1 server.go:254] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1210 10:01:35.057946       1 server.go:282] sriov_dpu device plugin endpoint started serving
I1210 10:01:35.057975       1 main.go:79] All servers started.
I1210 10:01:35.057981       1 main.go:80] Listening for term signals
I1210 10:01:35.714697       1 server.go:116] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1210 10:01:35.714778       1 server.go:157] ListAndWatch(sriov_dpu) invoked
I1210 10:01:35.714798       1 server.go:170] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},},}

But use pciAddresses can't find resources.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-6w57p
I1210 10:10:19.221745       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1210 10:10:19.237750       1 main.go:46] resource manager reading configs
I1210 10:10:19.237915       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "pciAddresses": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1210 10:10:19.238234       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0002a0e00]
I1210 10:10:19.238247       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc00041c840 AdditionalInfo:map[] SelectorObjs:[0xc0002a0e00]}]
I1210 10:10:19.238384       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1210 10:10:19.238392       1 main.go:62] Discovering host devices
I1210 10:10:19.333593       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1210 10:10:19.334017       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1210 10:10:19.335102       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1210 10:10:19.335192       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1210 10:10:19.335254       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1210 10:10:19.335316       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1210 10:10:19.335373       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1210 10:10:19.335435       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1210 10:10:19.335492       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1210 10:10:19.335594       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1210 10:10:19.335626       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335656       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335668       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335680       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335692       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335704       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335717       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335731       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335742       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335770       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335796       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1210 10:10:19.335811       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1210 10:10:19.335818       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1210 10:10:19.335822       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1210 10:10:19.335827       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1210 10:10:19.335832       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1210 10:10:19.335837       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1210 10:10:19.335841       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1210 10:10:19.335844       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1210 10:10:19.335847       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1210 10:10:19.335850       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335855       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335858       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335861       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335865       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335869       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335873       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335876       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335879       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335886       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335892       1 main.go:68] Initializing resource servers
I1210 10:10:19.335896       1 manager.go:117] number of config: 1
I1210 10:10:19.335900       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1210 10:10:19.335903       1 manager.go:122] DeviceType: netDevice
I1210 10:10:19.336001       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336256       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336504       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336782       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337072       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337408       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337750       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.338911       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.339158       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.339575       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.345010       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.349783       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.355430       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.355765       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.362164       1 manager.go:138] initServers(): selector index 0 will register 0 devices
I1210 10:10:19.362186       1 manager.go:142] no devices in device pool, skipping creating resource server for sriov_dpu
I1210 10:10:19.362191       1 main.go:74] Starting all servers...
I1210 10:10:19.362196       1 main.go:79] All servers started.
I1210 10:10:19.362199       1 main.go:80] Listening for term signals

@SchSeba
Copy link
Collaborator

SchSeba commented Dec 12, 2023

ok so I think we are good to just use the rootDevices right?

@SchSeba
Copy link
Collaborator

SchSeba commented Aug 19, 2024

closing this one

@SchSeba SchSeba closed this as completed Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix can't get pfname for virtio pci path JasinlikeEatingOrange/sriov-network-device-plugin
3 participants
@SchSeba @adrianchiris @JasinlikeEatingOrange