SPIKE: Research ways to update / rotate credentials for Software Factory

[Racer159]

Is your feature request related to a problem? Please describe.

As Ezra I want a way to rotate secrets consistently so that I can easily rotate creds for applications.

Describe the solution you'd like

We should explore different ways to do this:

• External Secrets Operator
• Pepr
• zarf tools update-creds?

Additional context

This should be simple from a user perspective and ideally be a one-stop-shop to do this.

[zachariahmiller]

Some notes:

Secret manager Integration for rotation is not available in gov cloud with RDS
Secrets Manager integration - Amazon Relational Database Service
View the AWS Region and Amazon RDS DB engine version availability for AWS Secrets Manager integration.

As far as I can tell a custom lambda is required for elasticache as there is no "integration" either. To manage such an integration would require enabling lambda in the account as well as custom code to support the rotation procedures and preventing application downtime and kubeapi access from the lambda to update the secrets and bounce the pods to pick up the new ones.

Seems like sonarqube doenst work with irsa without using a community wrapper that was single dev, years ago not updated (hopefully there's a better option)
Sonar Community
Sonarqube AWS Rds authentication through IAM role

There are problems with using IRSA for postgres databases
https://github.com/defenseunicorns/uds-prod-infrastructure/issues/171

We should honestly look at pepr for anything we cannot do with IRSA i would rather not have to maintain lambdas, images and build processes, etc.

[Racer159]

Pepr itself may be able to have IRSA if we wanted as well: https://github.com/defenseunicorns/zarf-init-aws

[Racer159]

For GitLab's RDS implementation here are the service accounts that would be impacted in the gitlab chart:

gitlab chart:

• gitlab.sidekiq.serviceAccount.annotations.irsa/role-arn - sets the ARN of the role for the sidekiq service
• gitlab.webservice.serviceAccount.annotations.irsa/role-arn - sets the ARN of the role for the webservice service
• gitlab.toolbox.serviceAccount.annotations.irsa/role-arn - sets the ARN of the role for the toolbox service
• gitlab.migrations.serviceAccount.annotations.irsa/role-arn - sets the ARN of the role for the migrations service
• gitlab.gitlab-exporter.serviceAccount.annotations.irsa/role-arn - sets the ARN of the role for the gitlab-exporter service

Note

For more information on how GitLab interacts with databases through its Helm chart see the GitLab documentation on connecting to external databases.

Would at least be a good starting point to kick this off