[Bug]: keytab cannot be provisioned when pod does not assign service and the value of scope is pod

[whg517]

Describe the bug

When a pod has no service, if no scope is configured in its secret pvc annotation or the scope is only a pod, the pod will not start properly.

The logic for secret-csi to generate keytab via pvc is: pod uses a temporary volume to make the secret storage class supply keytabs. After receiving the parameters configured in the pvc annotation, secret-CSI first creates the corresponding principals in kerberos and then generates keytabs containing these principals.

There is a special logic when generating a principal for a pod, when scope is a pod, and if the pod has a service association, then the hostname of the principal (e.g. foo/<svc-name>.svc.cluster.local@EXAMPLE.COM) is svc-name, and if there is no service, pod ip will be tried.

The key point is here, the logic of the pod is that the resource association is complete, the pod will start, and assign ip, if pvc is not ready, then the pod will not have IP, so in the case of pod no service and scope is pod, It's impossible to get the pod ip.

In addition, this situation is also more special, first of all, pod no service is rare, or only in the case of client access.

The current logic causes the csi program logic to fail, rather than giving a warning or providing an empty file to allow the pod to run properly, so this behavior is a BUG for secret-operator and csi.

What we need to discuss is that it is possible to use keytab only when the pod has no service. How should we deal with it? Whether csi is allowed to supply keytab for this situation, or to supply keytab normally with additional parameters, or to allow principals without host names to exist.

Have you searched existing issues? 🔎

I have searched the existing issues

Reproduction

apiVersion: v1
kind: Pod
metadata:
  name: krb5-pod-scope
  labels:
    name: krb5-pod-scope
spec:
  containers:
  - name: main
    image: rockylinux/rockylinux:9
    command:
    - "sh"
    - "-c"
    - |
      set -ex
      dnf install krb5-workstation nginx -y
      
      echo "Loop is running... (Press Ctrl+C or send SIGTERM to exit)"
      while ! test -f /opt/zncdata/keytab; do
        sleep 1
        echo "Waiting for content..."
      done

      KERBEROS_REALM=$(grep -oP 'default_realm = \K.*' /opt/zncdata//krb5.conf)

      klist -kt /opt/zncdata/keytab
      
      kinit -kt /opt/zncdata/keytab foo/krb5-pod-scope-service.$NAMESPACE.svc.cluster.local@$KERBEROS_REALM
      kinit -kt /opt/zncdata/keytab foo/krb5-pod-scope.krb5-pod-scope-service.$NAMESPACE.svc.cluster.local@$KERBEROS_REALM

      klist -e
      
      echo start nginx server
      nginx -g "daemon off;"
    env:
    - name: NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
    - name: NODE_NAME
      valueFrom:
        fieldRef:
          fieldPath: spec.nodeName
    - name: KRB5_CONFIG
      value: /opt/zncdata/krb5.conf
    resources:
      limits:
        memory: "128Mi"
        cpu: "500m"
    ports:
    - containerPort: 80
      protocol: TCP
      name: http

    volumeMounts:
    - name: secret
      mountPath: /opt/zncdata
    readinessProbe:
      httpGet:
        port: http
      initialDelaySeconds: 5
      periodSeconds: 5
  volumes:
  - name: secret
    ephemeral:
      volumeClaimTemplate:
        metadata:
          annotations:
            secrets.zncdata.dev/class: kerberos
            secrets.zncdata.dev/kerberosServiceNames: foo
            # secrets.zncdata.dev/scope: pod
        spec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: "1"
          storageClassName: secrets.zncdata.dev
          volumeMode: Filesystem

Screenshot

No response

Logs

No response

System Info

- go: go1.22.4 darwin/arm64
- secret-operator: v0.0.1
- secret-csi: v0.0.1

Severity

I can work around it

[whg517]

A solution that specifies headless services for Pods. When a pod is created, the specified subdomain is displayed, or a subdomain is specified for the pod in the pod template for deployment.

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: v1
kind: Pod
metadata:
  name: web-0
  labels:
    app: nginx
spec:
  subdomain: nginx
  containers:
  - name: nginx
    image: nginx

[whg517]

ref: #188