Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider: Introduce shared IAM propagation timeout, refactor KMS Key creation to that timeout and add KMS Key deletion to internal waiter package #12863

Merged
merged 1 commit into from
Apr 29, 2020

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Apr 16, 2020

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #7646
Reference: #12840

Release note for CHANGELOG:

* resource/aws_kms_external_key: Prevent `MalformedPolicyDocumentException` errors on creation by retrying for up to 2 minutes to wait for IAM change propagation
* resource/aws_kms_key: Prevent `MalformedPolicyDocumentException` errors on creation by retrying for up to 2 minutes to wait for IAM change propagation

The Terraform AWS Provider codebase contains many varied timeouts for handling IAM propagation retries. Here we introduce a shared constant for this amount of time. The choice of 2 minutes is based on that amount of time being:

  • Most widely used across resources
  • Based on lack of historical bug reports across those resources that implement that amount of time, most successful
  • Ensuring a reasonable user experience (not waiting too long) should there be an actual misconfiguration

As an initial implementation of this IAM propagation timeout and further showing the potential waiter package refactoring, this fixes shorter IAM timeout implementations in the aws_kms_key and aws_kms_external_key resources, while also refactoring the pending deletion logic. This second change is designed as an inflection point for how we want to handle imports across multiple waiter packages, with the preference of this initial implementation to name the Go import of the outside service, iamwaiter, or generically SERVICEwaiter. If agreed, this will be added to the proposal and the refactoring documentation.

NOTE: There is other StateChangeConf / StateRefreshFunc logic in these KMS resources, but this change is solely focused on highlighting the multiple import situation, and those will be handled later.

Output from acceptance testing:

--- PASS: TestAccAWSKmsExternalKey_basic (19.53s)
--- PASS: TestAccAWSKmsExternalKey_DeletionWindowInDays (31.61s)
--- PASS: TestAccAWSKmsExternalKey_Description (32.11s)
--- PASS: TestAccAWSKmsExternalKey_disappears (13.84s)
--- PASS: TestAccAWSKmsExternalKey_Enabled (312.55s)
--- PASS: TestAccAWSKmsExternalKey_KeyMaterialBase64 (104.29s)
--- PASS: TestAccAWSKmsExternalKey_Policy (33.78s)
--- PASS: TestAccAWSKmsExternalKey_Tags (43.70s)
--- PASS: TestAccAWSKmsExternalKey_ValidTo (165.77s)

--- PASS: TestAccAWSKmsKey_asymmetricKey (18.20s)
--- PASS: TestAccAWSKmsKey_basic (21.13s)
--- PASS: TestAccAWSKmsKey_disappears (13.92s)
--- PASS: TestAccAWSKmsKey_isEnabled (236.91s)
--- PASS: TestAccAWSKmsKey_policy (35.34s)
--- PASS: TestAccAWSKmsKey_Policy_IamRole (34.14s)
--- PASS: TestAccAWSKmsKey_Policy_IamServiceLinkedRole (44.80s)
--- PASS: TestAccAWSKmsKey_tags (34.65s)

…creation to that timeout and add KMS Key deletion to internal waiter package

Reference: #7646
Reference: #12840

The Terraform AWS Provider codebase contains many varied timeouts for handling IAM propagation retries. Here we introduce a shared constant for this amount of time. The choice of 2 minutes is based on that amount of time being:

- Most widely used across resources
- Based on lack of historical bug reports across those resources that implement that amount of time, most successful
- Ensuring a reasonable user experience (not waiting too long) should there be an actual misconfiguration

As an initial implementation of this IAM propagation timeout and further showing the potential waiter package refactoring, this fixes shorter IAM timeout implementations in the `aws_kms_key` and `aws_kms_external_key` resources, while also refactoring the pending deletion logic. This second change is designed as an inflection point for how we want to handle imports across multiple waiter packages, with the preference of this initial implementation to name the Go import of the outside service, `iamwaiter`, or generically SERVICEwaiter. If agreed, this will be added to the proposal and the refactoring documentation.

NOTE: There is other `StateChangeConf` / `StateRefreshFunc` logic in these KMS resources, but this change is solely focused on highlighting the multiple import situation, and those will be handled later.

Output from acceptance testing:

```
--- PASS: TestAccAWSKmsExternalKey_basic (19.53s)
--- PASS: TestAccAWSKmsExternalKey_DeletionWindowInDays (31.61s)
--- PASS: TestAccAWSKmsExternalKey_Description (32.11s)
--- PASS: TestAccAWSKmsExternalKey_disappears (13.84s)
--- PASS: TestAccAWSKmsExternalKey_Enabled (312.55s)
--- PASS: TestAccAWSKmsExternalKey_KeyMaterialBase64 (104.29s)
--- PASS: TestAccAWSKmsExternalKey_Policy (33.78s)
--- PASS: TestAccAWSKmsExternalKey_Tags (43.70s)
--- PASS: TestAccAWSKmsExternalKey_ValidTo (165.77s)

--- PASS: TestAccAWSKmsKey_asymmetricKey (18.20s)
--- PASS: TestAccAWSKmsKey_basic (21.13s)
--- PASS: TestAccAWSKmsKey_disappears (13.92s)
--- PASS: TestAccAWSKmsKey_isEnabled (236.91s)
--- PASS: TestAccAWSKmsKey_policy (35.34s)
--- PASS: TestAccAWSKmsKey_Policy_IamRole (34.14s)
--- PASS: TestAccAWSKmsKey_Policy_IamServiceLinkedRole (44.80s)
--- PASS: TestAccAWSKmsKey_tags (34.65s)
```
@bflad bflad added the bug Addresses a defect in current functionality. label Apr 16, 2020
@bflad bflad requested a review from a team April 16, 2020 18:13
@ghost ghost added size/XL Managed by automation to categorize the size of a PR. needs-triage Waiting for first response or review from a maintainer. service/kms Issues and PRs that pertain to the kms service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Apr 16, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Apr 17, 2020
Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

--- PASS: TestAccAWSKmsKey_disappears (9.06s)
--- PASS: TestAccAWSKmsExternalKey_disappears (9.53s)
--- PASS: TestAccAWSKmsKey_asymmetricKey (11.22s)
--- PASS: TestAccAWSKmsExternalKey_basic (11.69s)
--- PASS: TestAccAWSKmsKey_basic (12.24s)
--- PASS: TestAccAWSKmsExternalKey_Policy (14.57s)
--- FAIL: TestAccAWSKmsExternalKey_Description (15.82s)
--- PASS: TestAccAWSKmsExternalKey_DeletionWindowInDays (16.21s)
--- PASS: TestAccAWSKmsKey_policy (16.63s)
--- PASS: TestAccAWSKmsKey_tags (16.94s)
--- PASS: TestAccAWSKmsExternalKey_Tags (21.42s)
--- PASS: TestAccAWSKmsKey_Policy_IamRole (28.56s)
--- PASS: TestAccAWSKmsKey_Policy_IamServiceLinkedRole (39.46s)
--- PASS: TestAccAWSKmsExternalKey_KeyMaterialBase64 (72.37s)
--- FAIL: TestAccAWSKmsExternalKey_ValidTo (203.10s)
--- PASS: TestAccAWSKmsExternalKey_Enabled (237.38s)
--- PASS: TestAccAWSKmsKey_isEnabled (245.15s)

Looks like it's hitting some pre-existing eventual consistency errors. Nothing introduced by this PR.

@bflad bflad added this to the v2.60.0 milestone Apr 29, 2020
@bflad bflad merged commit 49914d3 into master Apr 29, 2020
@bflad bflad deleted the b-aws_kms_key-MalformedPolicyDocumentException branch April 29, 2020 23:31
bflad added a commit that referenced this pull request Apr 29, 2020
@ghost
Copy link

ghost commented May 1, 2020

This has been released in version 2.60.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

adamdecaf pushed a commit to adamdecaf/terraform-provider-aws that referenced this pull request May 28, 2020
…creation to that timeout and add KMS Key deletion to internal waiter package (hashicorp#12863)

Reference: hashicorp#7646
Reference: hashicorp#12840

The Terraform AWS Provider codebase contains many varied timeouts for handling IAM propagation retries. Here we introduce a shared constant for this amount of time. The choice of 2 minutes is based on that amount of time being:

- Most widely used across resources
- Based on lack of historical bug reports across those resources that implement that amount of time, most successful
- Ensuring a reasonable user experience (not waiting too long) should there be an actual misconfiguration

As an initial implementation of this IAM propagation timeout and further showing the potential waiter package refactoring, this fixes shorter IAM timeout implementations in the `aws_kms_key` and `aws_kms_external_key` resources, while also refactoring the pending deletion logic. This second change is designed as an inflection point for how we want to handle imports across multiple waiter packages, with the preference of this initial implementation to name the Go import of the outside service, `iamwaiter`, or generically SERVICEwaiter. If agreed, this will be added to the proposal and the refactoring documentation.

NOTE: There is other `StateChangeConf` / `StateRefreshFunc` logic in these KMS resources, but this change is solely focused on highlighting the multiple import situation, and those will be handled later.

Output from acceptance testing:

```
--- PASS: TestAccAWSKmsExternalKey_basic (19.53s)
--- PASS: TestAccAWSKmsExternalKey_DeletionWindowInDays (31.61s)
--- PASS: TestAccAWSKmsExternalKey_Description (32.11s)
--- PASS: TestAccAWSKmsExternalKey_disappears (13.84s)
--- PASS: TestAccAWSKmsExternalKey_Enabled (312.55s)
--- PASS: TestAccAWSKmsExternalKey_KeyMaterialBase64 (104.29s)
--- PASS: TestAccAWSKmsExternalKey_Policy (33.78s)
--- PASS: TestAccAWSKmsExternalKey_Tags (43.70s)
--- PASS: TestAccAWSKmsExternalKey_ValidTo (165.77s)

--- PASS: TestAccAWSKmsKey_asymmetricKey (18.20s)
--- PASS: TestAccAWSKmsKey_basic (21.13s)
--- PASS: TestAccAWSKmsKey_disappears (13.92s)
--- PASS: TestAccAWSKmsKey_isEnabled (236.91s)
--- PASS: TestAccAWSKmsKey_policy (35.34s)
--- PASS: TestAccAWSKmsKey_Policy_IamRole (34.14s)
--- PASS: TestAccAWSKmsKey_Policy_IamServiceLinkedRole (44.80s)
--- PASS: TestAccAWSKmsKey_tags (34.65s)
```
@ghost
Copy link

ghost commented May 30, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators May 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/kms Issues and PRs that pertain to the kms service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Eventual consistency issue with aws_iam_service_linked_role
2 participants