When mounting two PVCs from EFS into a single deployment, Kubernetes pods get stuck in the 'ContainerCreating' state with 0/1

[jorgenivia]

/kind bug

What happened?

When configuring two EFS PVCs in a deployment, it gets stuck in the 'ContainerCreating' state with 0/1. However, when configuring only one EFS PVC in the deployment, the pod starts correctly.

What you expected to happen?

When mounting two or more EFS PVCs in a single deployment, the pod will start correctly.

How to reproduce it (as minimally and precisely as possible)?

Associate two EFS volumes with a pod and deploy to demonstrate the error.

---
apiVersion: v1
kind: Pod
metadata:
  name: efs-app
spec:
  containers:
  - name: app
    image: centos
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data    
    - name: persistent-storage2
      mountPath: /data2
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim
  - name: persistent-storage2
    persistentVolumeClaim:
      claimName: efs-claim-2

Anything else we need to know?:

When executing the command 'kubectl describe pod', we only see an event: 'Successfully assigned namespace/xxx-xxx-x to ip-xxx-xx-xx-x.us-xxx-x.compute.internal'.

Environment

• Kubernetes version (use kubectl version): v1.29.2
• Driver version: v2.0.2-eksbuild.1

[mskanth972]

Hi @jorgenivia, I am able to mount two volumes successfully

[mskanth@dev-dsk-mskanth-1e-96a9e8cf AddonTest]$ k get pvc
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
efs-claim-1   Bound    pvc-94a17c4f-c826-4d25-bc3b-74c5d030431f   5Gi        RWX            efs-sc         14s
efs-claim-2   Bound    pvc-f1c3ffa9-8bf7-44bd-bc7d-96c94419ff5a   5Gi        RWX            efs-sc         13s

Can you share me the PVC configuration file and also details of your setup(installation and deployment steps)

[jorgenivia]

I am sharing the configuration of the storage class, the PVs, and the PVCs, as well as the pod.

[jorgenivia]

I am sharing the configuration of the storage class, the PVs, and the PVCs, as well as the pod.

storage class

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: aws-efs
  fileSystemId: fs-xxxxxxxxxxxxx
  directoryPerms: "777"
  #uid: "0"
  #gid: "0"
  gidRangeStart: "1000" # optional
  gidRangeEnd: "2000" # optional
  basePath: "/efs" # optional
  #subPathPattern: "${.PVC.namespace}/${.PVC.name}" # optional
  ensureUniqueDirectory: "true" # optional
  reuseAccessPoint: "false" # optional

pv - pvc -pod

apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-claim
  labels:
    app: docs-ms-xxxxxxxxxxx
spec:
  capacity:
    storage: 20Gi
  accessModes:
  - ReadWriteMany
  storageClassName: aws-efs
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-xxxxxxxxxxxxx
    volumeAttributes:
      path: /efs/efs-claim
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: efs-claim
  namespace: default
  labels:
    app: docs-ms-xxxxxxxxxxx
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: aws-efs
  selector:
    matchLabels:
      app: docs-ms-xxxxxxxxxxx
  resources:
    requests:
      storage: 20Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-claim-2
  labels:
    app: docs-ms-2-xxxxxxxxxxx
spec:
  capacity:
    storage: 10Gi
  accessModes:
  - ReadWriteMany
  storageClassName: aws-efs
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-xxxxxxxxxxxxx
    volumeAttributes:
      path: /efs/efs-claim-2
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: efs-claim-2
  namespace: default
  labels:
    app: docs-ms-2-xxxxxxxxxxx
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: aws-efs
  selector:
    matchLabels:
      app: docs-ms-2-xxxxxxxxxxx
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: efs-app
spec:
  containers:
  - name: app
    image: centos
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
    - name: persistent-storage2
      mountPath: /data2
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim
  - name: persistent-storage2
    persistentVolumeClaim:
      claimName: efs-claim-2

[mskanth972]

In Dynamic Provisioning Once you create the PVC, the Kubernetes API server interacts with the EFS CSI Driver based on the configured StorageClass. The EFS CSI Driver dynamically provisions a PersistentVolume (PV) on your behalf, you dont need to specify PV separately as above. Refer my yaml example below.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim-1
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim-2
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: efs-app
spec:
  containers:
    - name: app
      image: centos
      command: ["/bin/sh"]
      args: ["-c", "while true; do echo $(date -u) >> /data1/out; sleep 5; done && while true; do echo $(date -u) >> /data2/out; sleep 5; done"]
      volumeMounts:
        - name: persistent-storage-1
          mountPath: /data1
        - name: persistent-storage-2
          mountPath: /data2
  volumes:
    - name: persistent-storage-1
      persistentVolumeClaim:
        claimName: efs-claim-1
    - name: persistent-storage-2
      persistentVolumeClaim:
        claimName: efs-claim-2