pxc-db pmm configure pmmserverkey

[weiss-m]

We use the pxc operator password auto generation feature for pxc accounts like root, xtrabackup ...

Unfortunately, pxc operator can't auto generate pmmserverkey.
So we have to add the pmmserverkey manually to the existing k8s secret which already contains the auto generated passwords for root,xtrabackup ...
https://github.com/percona/percona-xtradb-cluster-operator/blob/main/deploy/secrets.yaml#L11
https://docs.percona.com/percona-operator-for-mysql/pxc/monitoring.html#install-pmm-server

This is a manual step which we would like to avoid. We are using argocd for deployment of several pxc clusters.
So, one option would be, to use kustomize to add pmmserverkey to the existing k8s secret. This is not an option for us because we don't store passwords in clear text in git.
We would like to use sealed secrets, but sealed secrets doesn't support addition of a key, value pair to an existing k8s secret.

Would it be possible to store auto generated and manual created credentials in separate k8s secrets?

[spron-in]

Hey @weiss-m ,

so let me see if I understand the problem:

1. You deploy PXC with argocd
2. You rely on auto-generated secrets for MySQL
3. At the same time you use PMM and you don't have an easy automated way to add pmmserverkey into the generated secret.

Let me think how we can do that.
@hors @egegunes any thoughts?

[weiss-m]

Just for my understanding, is the following assumption correct? The operator uses cluster-secrets as input and applies the password accordingly to cluster-secrets in the database and afterward applying it writes the passwords in secret internal-cluster ?
So in cluster-secrets is always the desired state? And in internal-cluster the current state?

[spron-in]

Yes, you are correct.

[ydixken]

Also raising this issue with the Percona Support, as we currently have to configure pmm manually (create the API key) on cluster creation, there should be a way to provide a pre-seeded api key.