SecretProviderClass should be versioned similar to configmaps

[fishy]

Describe the solution you'd like

Let's say we have a service A, which currently depends on 2 secrets, B & C, provided by a SecretProviderClass.

Then, a new version of A removed the usage of secret C, so it now only need one secret B.

When deploying the new version, 2 things will happen:

1. The SecretProviderClass being updated to remove C and only provide B
2. The pods for A being gradually replaced by the new version that no longer needs C

But since 1 is almost immediate while 2 is always a gradual rollout, after 1 is replaced the old pods still requiring both B and C will likely start to fail because the update of SecretProviderClass that removed C that they still require.

Even if we tweak the order to only do 1 after 2 is fully done, it can still cause the reverse problem (a new version that requires a new secret).

Currently, to avoid this issue we need to be extra careful and do 2-step deploys, first deploy a superset of SecretProviderClass that contains the secrets for both new and old version of code with the new code, and after that's fully done, do another deploy to shrink SecretProviderClass to only the ones needed by the new version.

But I think to fully address this problem, we should make SecretProviderClass versioned (similar to how configmaps are versioned), so every pod can use their matching version of SecretProviderClass to provide the secrets they need.

Anything else you would like to add:

(ported from kubernetes/kubernetes#123955)

Environment:

• Secrets Store CSI Driver version: (use the image tag):
• Kubernetes version: (use kubectl version): 1.27.10

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[aramase]

/remove-lifecycle stale
/lifecycle frozen