cephfs static volume credentials expect userID instead of adminID (mount failed)

[hashmap-kz]

When creating a static volume with PV/PVC for cephfs it fails with error: "failed to get user credentials from node stage secrets: missing ID field 'userID' in secrets".

But: according to helm chart we creating the secret for cephfs-csi using adminID and adminKey credentials: * https://github.com/ceph/ceph-csi/blob/devel/charts/ceph-csi-cephfs/values.yaml#L327

When I change nodeStageSecretRef.name to csi-rbd-secret and nodeStageSecretRef.namespace to ceph-csi-rbd (where my secrets for ceph-csi-rbd are placed) - then everything works fine.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: cephfs-static-pv
spec:
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: csi-cephfs-secret
      namespace: ceph-csi-cephfs
    volumeAttributes:
      "fsName": "cephfs"
      "clusterID": "375d6ad4-01bd-4492-acd7-..."
      "staticVolume": "true"
      "rootPath": /volumes/backup
    volumeHandle: cephfs-static-pv
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

A clear and concise description of what the bug is.

Environment details

• Image/version of Ceph CSI driver : quay.io/cephcsi/cephcsi:v3.10.1
• Helm chart version : 3.10.1
• Kernel version : 6.1.0-18-amd64
• Mounter used for mounting PVC (for cephFS its fuse or kernel. for rbd its
  krbd or rbd-nbd) : kernel
• Kubernetes cluster version : 1.28.2
• Ceph cluster version : reef

Steps to reproduce

Steps to reproduce the behavior:

1. Setup details:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: cephfs-static-pv
spec:
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: csi-cephfs-secret
      namespace: ceph-csi-cephfs
    volumeAttributes:
      "fsName": "cephfs"
      "clusterID": "375d6ad4-01bd-4492-acd7-..."
      "staticVolume": "true"
      "rootPath": /volumes/backup
    volumeHandle: cephfs-static-pv
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cephfs-static-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: ""
  volumeMode: Filesystem
  volumeName: cephfs-static-pv

---
apiVersion: v1
kind: Pod
metadata:
  name: csi-cephfs-demo-pod
spec:
  containers:
    - name: web-server
      image: docker.io/library/nginx:latest
      volumeMounts:
        - name: mypvc
          mountPath: /var/lib/www
      resources: {}
  volumes:
    - name: mypvc
      persistentVolumeClaim:
        claimName: cephfs-static-pvc
        readOnly: false

2. See error

Containers:
  web-server:
    Container ID:
    Image:          docker.io/library/nginx:latest
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/lib/www from mypvc (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-99tnz (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  mypvc:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  cephfs-static-pvc
    ReadOnly:   false
  kube-api-access-99tnz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  5s (x2 over 19s)  default-scheduler  0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling..
  Normal   Scheduled         2s                default-scheduler  Successfully assigned default/csi-cephfs-demo-pod to w2-k8s.cmp.com
  Warning  FailedMount       0s (x2 over 1s)   kubelet            MountVolume.MountDevice failed for volume "cephfs-static-pv" : rpc error: code = Internal desc = failed to get user credentials from node stage secrets: missing ID field 'userID' in secrets

[Madhu-1]

For static PVC the userID and userKey are expected values in secret. if they are not present it fails.

[hashmap-kz]

Thank you.