Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redshift cluster AZ relocation support #20812

Conversation

nickolivera
Copy link
Contributor

@nickolivera nickolivera commented Sep 7, 2021

Updated for new structure
Adds support for Cluster Relocation.

https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-redshift-launches-ability-easily-move-clusters-between-aws-availability-zones/
https://github.com/aws/aws-sdk-go/blob/main/models/apis/redshift/2012-12-01/api-2.json#L2432
https://github.com/aws/aws-sdk-go/blob/main/models/apis/redshift/2012-12-01/api-2.json#L2952
https://github.com/aws/aws-sdk-go/blob/main/models/apis/redshift/2012-12-01/api-2.json#L4904
https://github.com/aws/aws-sdk-go/blob/main/models/apis/redshift/2012-12-01/api-2.json#L5614

Notes for the tests:

  • You can't disable automated snapshots for ra3.xlplus node types. Set the automated retention period from 1-35 days.
  • Availability zone relocation feature is not supported for publicly accessible cluster.
  • Availability zone relocation feature is not supported for non ra3 node type.
  • The Redshift Subnet Group must have more than one AZ available

Notes for the API:

AvailabilityZoneRelocation is not available when describing clusters, I've used AvailabilityZoneRelocationStatus and inferring the status of the former for plans and imports. The API is official but not documented properly, but it seems to have 4 statuses from my own research, I've added a fallback to the state and default if the API returns something else. I would love to have feedback regarding that piece of the implementation, and advice for how to structure properly (if not) when the reads from the API are not 1:1 translatable to create/update.

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #19098

Output from acceptance testing:


TF_ACC=1 go test ./internal/service/redshift -v -count 1 -parallel 3 -run='TestAccRedshiftCluster_availabilityZoneRelocation' -timeout=180m
TF_ACC=1 go test ./internal/service/redshift -v -count 1 -parallel 3 -run='TestAccRedshiftClusterDataSource_basic' -timeout=180m

nick@nick terraform-provider-aws % TF_ACC=1 go test ./internal/service/redshift -v -count 1 -parallel 3 -run='TestAccRedshiftClusterDataSource_basic' -timeout=180m
=== RUN   TestAccRedshiftClusterDataSource_basic
=== PAUSE TestAccRedshiftClusterDataSource_basic
=== CONT  TestAccRedshiftClusterDataSource_basic
--- PASS: TestAccRedshiftClusterDataSource_basic (446.67s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/redshift   449.299s
nick@nick terraform-provider-aws % TF_ACC=1 go test ./internal/service/redshift -v -count 1 -parallel 3 -run='TestAccRedshiftCluster_availabilityZoneRelocation' -timeout=180m
=== RUN   TestAccRedshiftCluster_availabilityZoneRelocation
=== PAUSE TestAccRedshiftCluster_availabilityZoneRelocation
=== CONT  TestAccRedshiftCluster_availabilityZoneRelocation
--- PASS: TestAccRedshiftCluster_availabilityZoneRelocation (535.35s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/redshift   537.962s

@github-actions github-actions bot added documentation Introduces or discusses updates to documentation. service/redshift Issues and PRs that pertain to the redshift service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. needs-triage Waiting for first response or review from a maintainer. size/M Managed by automation to categorize the size of a PR. labels Sep 7, 2021
@breathingdust breathingdust added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Sep 8, 2021
@nickolivera
Copy link
Contributor Author

I've reimplemented/simplified the read functionality based on how other resources already implemented are dealing with the same scenario.

user:terraform-provider-aws user$ make testacc TESTARGS="-run=TestAccAWSRedshiftCluster_availabilityZoneRelocation"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSRedshiftCluster_availabilityZoneRelocation -timeout 180m
=== RUN   TestAccAWSRedshiftCluster_availabilityZoneRelocation
=== PAUSE TestAccAWSRedshiftCluster_availabilityZoneRelocation
=== CONT  TestAccAWSRedshiftCluster_availabilityZoneRelocation
--- PASS: TestAccAWSRedshiftCluster_availabilityZoneRelocation (428.04s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       428.137s
user:terraform-provider-aws user$ make testacc TESTARGS="-run=TestAccAWSRedshiftCluster_basic" ACCTEST_PARALLELISM=1
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 1 -run=TestAccAWSRedshiftCluster_basic -timeout 180m
=== RUN   TestAccAWSRedshiftCluster_basic
=== PAUSE TestAccAWSRedshiftCluster_basic
=== CONT  TestAccAWSRedshiftCluster_basic
--- PASS: TestAccAWSRedshiftCluster_basic (315.29s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       315.396s

@zhelding
Copy link
Contributor

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

@github-actions github-actions bot added the pre-service-packages Includes pre-Service Packages aspects. label Dec 13, 2021
@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. and removed size/M Managed by automation to categorize the size of a PR. pre-service-packages Includes pre-Service Packages aspects. labels Dec 13, 2021
@github-actions github-actions bot added size/M Managed by automation to categorize the size of a PR. and removed size/XL Managed by automation to categorize the size of a PR. labels Dec 13, 2021
Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this, @nickyamanaka. 🚀

I've made a couple changes to the PR:

  • availability_zone_relocation and availability_zone_relocation_status are essentially the same field, one for input and one for output because it can be in a pending state. I've combined them and renamed availability_zone_relocation to availability_zone_relocation_enabled for clarity
  • Since availability_zone_relocation enables relocation, but doesn't cause a relocation, I've added tests for relocating the cluster instead of recreating the cluster as it did previously. I also discovered that no other modifications can be made during relocation, so I updated that as well.

--- PASS: TestAccRedshiftCluster_availabilityZoneRelocation_publiclyAccessible (78.25s)
--- PASS: TestAccRedshiftCluster_disappears (408.94s)
--- PASS: TestAccRedshiftClusterDataSource_basic (422.33s)
--- PASS: TestAccRedshiftCluster_basic (456.92s)
--- PASS: TestAccRedshiftClusterDataSource_logging (464.35s)
--- PASS: TestAccRedshiftCluster_kmsKey (500.20s)
--- PASS: TestAccRedshiftCluster_snapshotCopy (510.77s)
--- PASS: TestAccRedshiftCluster_withFinalSnapshot (524.53s)
--- PASS: TestAccRedshiftCluster_loggingEnabled (531.72s)
--- PASS: TestAccRedshiftCluster_forceNewUsername (551.24s)
--- PASS: TestAccRedshiftCluster_iamRoles (556.87s)
--- PASS: TestAccRedshiftCluster_tags (605.85s)
--- PASS: TestAccRedshiftCluster_enhancedVPCRoutingEnabled (618.59s)
--- PASS: TestAccRedshiftCluster_publiclyAccessible (622.10s)
--- PASS: TestAccRedshiftClusterDataSource_availabilityZoneRelocationEnabled (572.44s)
--- PASS: TestAccRedshiftClusterDataSource_vpc (277.32s)
--- PASS: TestAccRedshiftCluster_availabilityZoneRelocation (780.50s)
--- PASS: TestAccRedshiftCluster_changeAvailabilityZone_availabilityZoneRelocationNotSet (420.35s)
--- PASS: TestAccRedshiftCluster_changeAvailabilityZone (833.09s)
--- PASS: TestAccRedshiftCluster_changeAvailabilityZoneAndSetAvailabilityZoneRelocation (676.53s)
--- PASS: TestAccRedshiftCluster_changeEncryption2 (1303.43s)
--- PASS: TestAccRedshiftCluster_updateNodeType (1340.54s)
--- PASS: TestAccRedshiftCluster_updateNodeCount (1506.16s)
--- PASS: TestAccRedshiftCluster_changeEncryption1 (1855.43s)

@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. and removed size/M Managed by automation to categorize the size of a PR. labels Mar 12, 2022
@gdavison gdavison merged commit 29acd34 into hashicorp:main Mar 12, 2022
@github-actions github-actions bot added this to the v4.6.0 milestone Mar 12, 2022
@github-actions
Copy link

This functionality has been released in v4.6.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

github-actions bot commented May 7, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/redshift Issues and PRs that pertain to the redshift service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Amazon Redshift AZ Relocation
4 participants