Support for AWS::SecretsManager::SecretTargetAttachment

[ghost]

This issue was originally opened by pwong-rms as hashicorp/terraform#21903. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Current Terraform Version

Terraform v0.12.3
+ provider.aws v2.16.0

Use-cases

We are converting from CloudFormation to Terraform. We create a secret with username and password in AWS Secrets Manager, then create a RDS DB instance (not cluster), then use the secret target attachment to attach the secret to the DB instance. This then results in AWS populating the secret with additional information.

Then, we use a rotation schedule and lambda to update the secret. Because the secret is attached to the DB instance, it automatically changes the DB instance's password to match.

The example given here for "Creating a Secret and an RDS DB Instance" is basically what we have in CF: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-secretsmanager-secrettargetattachment.html

Attempted Solutions

Attempted solutions:

• Use a local file that keeps track of the date. Then use the file contents as a keeper for a random string resource. But, because the local file's contents are not known until after apply, it forces downstream resources to update every time.
• Use formatdate("YYYY-MM", timestamp()) as a keeper for the random string resource. But this too is not known until after apply.
• Change the secret rotation lambda so that it can directly change the password of the DB instance. Rather than only changing the secret, and expecting that to get propagated to the DB instance. This is currently considered too costly, although we may have to do it later.

Proposal

New resource type: aws_secretsmanager_secret_target_attachment - supporting all features described here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-secretsmanager-secrettargetattachment.html

Also a new input for aws_db_instance taking a secret's ID / ARN, to implicitly create the secret target attachment.

References

[pwong-rms]

Currently working around this by having the AWS::SecretsManager::Secret, AWS::SecretsManager::SecretTargetAttachment, and AWS::SecretsManager::RotationSchedule being deployed as separate CloudFormation stacks.

The secret itself is deployed as a CloudFormation stack because that causes its ID to be slightly randomized - thus allowing for testing by repeated apply and destroy

[smiller171]

@pwong-rms You can randomize the name with the uuid() function instead of cloudformation

[Hokwang]

@anGie44 Please re-consider this issue priority.

[a0s]

Any news on this?

[amirdamirov]

Also waiting for this feature. It really blocks normal automation with terraform.

[bushong1]

2023 Bump

[preshen-goobiah]

+1

[leogasparini]

+1

[sassdavid]

+1

[kaskad737]

+1

[danielwhite]

This may largely be addressed for RDS resources with the support for manage_master_user_password implemented by #28538.

[StianHanssen]

This may largely be addressed for RDS resources with the support for manage_master_user_password implemented by #28538.

I disagree. manage_master_user_password is not supported for all variations. For instance, Aurora Serverless v1 is not supported. I would very much like proper support for SecretTargetAttachment, not only for storing the username and password, but for all the other attributes such as the database name, port, etc.

[mvanbaak]

Yes, manage_master_user_password is a good step forwards, but it's only the master user password. I would like the SecretTargetAttachment to manage user credentials for services as well. Have it all in terraform makes things a LOT better.

[ameer308]

agreed - typically we prefer to have the whole set of information about a cluster that SecretTargetAttachment provides which includes endpoint/host information.
since the manage_master_user_password doesn't allow editing once created, we can't add this afterwards, so in effect, we're still creating the secret manually. this is our workflow:

1)create db w/ manually created secret:
2)import secret and version
3)create complete secret with endpoint and other info etc

[jar-b]

Hello everyone - after researching this issue we've composed a design decision document with our findings. The complete document is viewable in #34208, and the proposal section is included below for reference:

Proposal

Given the currently available AWS APIs, there isn’t a path to implement the workflow from the SecretTargetAttachment CloudFormation function in the AWS Terraform Provider. Existing resources provide options to either fully customize a manual secret with both database credentials and connection information, or to fully offload secret management to AWS. These options cover the core use case of storing and rotating database secrets via AWS SecretsManager.

With existing options already available in the provider, and no clear path forward the proposal is to close this issue.

Since we're proposing to close this issue, we're planning to leave this issue and the corresponding design document open for two weeks (through 11/15/2023) in order to allow time for community feedback. If you believe there is a viable option for implementing this functionality with publicly available APIs, please leave a review!

---

On a related note - while researching this issue we've added support for modifying managed secret rotation schedules and support for managed passwords with Redshift clusters. Both will be available in v5.24.0 of the provider, due later this week. The pull requests implementing these changes are below:

#34180
#34182

[jar-b]

Hello - We haven't received any objections to the proposal linked above, so we will be marking it as accepted and merging the design document. We will also close this issue, and recommend users pursue the methods outlined in the proposal to achieve this functionality with existing Terraform resources.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.