Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout support for EBS Snapshot copies #20912

Merged

Conversation

ahorner
Copy link
Contributor

@ahorner ahorner commented Sep 15, 2021

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Description

These copies can take an exorbitant amount of time to run, and if a
snapshot requires a custom timeout configuration, chances are
extremely high that any attempt to copy that snapshot will also exceed
the default 10-minute timeout.

Thankfully, the API is very similar between the base snapshot resource
and the copy resource, so we can borrow the logic almost wholesale
from the EBS snapshot to support this configuration block
appropriately.

Closes #14354.
Closes #17186.

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSEbsSnapshotCopy_basic'

TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEbsSnapshotCopy -timeout 180m
=== RUN   TestAccAWSEbsSnapshotCopy_basic
=== PAUSE TestAccAWSEbsSnapshotCopy_basic
=== RUN   TestAccAWSEbsSnapshotCopy_tags
=== PAUSE TestAccAWSEbsSnapshotCopy_tags
=== RUN   TestAccAWSEbsSnapshotCopy_withDescription
=== PAUSE TestAccAWSEbsSnapshotCopy_withDescription
=== RUN   TestAccAWSEbsSnapshotCopy_withRegions
=== PAUSE TestAccAWSEbsSnapshotCopy_withRegions
=== RUN   TestAccAWSEbsSnapshotCopy_withKms
=== PAUSE TestAccAWSEbsSnapshotCopy_withKms
=== RUN   TestAccAWSEbsSnapshotCopy_disappears
=== PAUSE TestAccAWSEbsSnapshotCopy_disappears
=== CONT  TestAccAWSEbsSnapshotCopy_basic
=== CONT  TestAccAWSEbsSnapshotCopy_withRegions
=== CONT  TestAccAWSEbsSnapshotCopy_withDescription
=== CONT  TestAccAWSEbsSnapshotCopy_disappears
=== CONT  TestAccAWSEbsSnapshotCopy_withKms
=== CONT  TestAccAWSEbsSnapshotCopy_tags
--- PASS: TestAccAWSEbsSnapshotCopy_basic (70.99s)
--- PASS: TestAccAWSEbsSnapshotCopy_withKms (71.78s)
--- PASS: TestAccAWSEbsSnapshotCopy_withDescription (71.83s)
--- PASS: TestAccAWSEbsSnapshotCopy_disappears (87.11s)
--- PASS: TestAccAWSEbsSnapshotCopy_tags (106.00s)
--- PASS: TestAccAWSEbsSnapshotCopy_withRegions (379.15s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	382.381s

These copies can take an exorbitant amount of time to run, and if a
snapshot requires a custom timeout configuration, chances are
extremely high that any attempt to copy that snapshot will also exceed
the default 10-minute timeout.

Thankfully, the API is very similar between the base snapshot resource
and the copy resource, so we can borrow the logic almost wholesale
from the EBS snapshot to support this configuration block
appropriately.

Resolves hashicorp#14354.
@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/ec2 Issues and PRs that pertain to the ec2 service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. size/M Managed by automation to categorize the size of a PR. labels Sep 15, 2021
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @ahorner 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@github-actions github-actions bot added the documentation Introduces or discusses updates to documentation. label Sep 16, 2021
@breathingdust breathingdust added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Sep 16, 2021
@devindice
Copy link

devindice commented Oct 2, 2021

You beat me to it! This was crippling my company after switching to encrypted snapshots.

To compile from source:
git clone https://github.com/hashicorp/terraform-provider-aws.git
cd terraform-provider-aws
Checkout version and add code from this commit
Install all dependancies: go mod vendor
Make the directory: mkdir -p ~/.terraform.d/plugins/your.domain/yourCompany/aws/3.42.0/linux_amd64/
Compile: go build -o terraform-provider-aws

mv terraform-provider-aws ~/.terraform.d/plugins/your.domain/yourCompany/aws/3.42.0/linux_amd64/terraform-provider-aws

Path/terraform/providers.tf:

terraform {
required_providers {
aws = {
source = "your.domain/yourCompany/aws"
version = "~> 3.0"
}
}
}

@zhelding
Copy link
Contributor

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

@github-actions github-actions bot added service/ec2ebs Issues and PRs that pertain to the ec2ebs service. size/XS Managed by automation to categorize the size of a PR. and removed tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. service/ec2 Issues and PRs that pertain to the ec2 service. size/M Managed by automation to categorize the size of a PR. labels Jun 5, 2022
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTARGS='-run=TestAccEC2EBSSnapshotCopy_' PKG=ec2 ACCTEST_PARALLELISM=2
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/ec2/... -v -count 1 -parallel 2  -run=TestAccEC2EBSSnapshotCopy_ -timeout 180m
=== RUN   TestAccEC2EBSSnapshotCopy_basic
=== PAUSE TestAccEC2EBSSnapshotCopy_basic
=== RUN   TestAccEC2EBSSnapshotCopy_disappears
=== PAUSE TestAccEC2EBSSnapshotCopy_disappears
=== RUN   TestAccEC2EBSSnapshotCopy_tags
=== PAUSE TestAccEC2EBSSnapshotCopy_tags
=== RUN   TestAccEC2EBSSnapshotCopy_withDescription
=== PAUSE TestAccEC2EBSSnapshotCopy_withDescription
=== RUN   TestAccEC2EBSSnapshotCopy_withRegions
=== PAUSE TestAccEC2EBSSnapshotCopy_withRegions
=== RUN   TestAccEC2EBSSnapshotCopy_withKMS
=== PAUSE TestAccEC2EBSSnapshotCopy_withKMS
=== RUN   TestAccEC2EBSSnapshotCopy_storageTier
=== PAUSE TestAccEC2EBSSnapshotCopy_storageTier
=== CONT  TestAccEC2EBSSnapshotCopy_basic
=== CONT  TestAccEC2EBSSnapshotCopy_withRegions
--- PASS: TestAccEC2EBSSnapshotCopy_basic (75.74s)
=== CONT  TestAccEC2EBSSnapshotCopy_withDescription
--- PASS: TestAccEC2EBSSnapshotCopy_withRegions (79.96s)
=== CONT  TestAccEC2EBSSnapshotCopy_tags
--- PASS: TestAccEC2EBSSnapshotCopy_withDescription (73.59s)
=== CONT  TestAccEC2EBSSnapshotCopy_disappears
--- PASS: TestAccEC2EBSSnapshotCopy_tags (105.01s)
=== CONT  TestAccEC2EBSSnapshotCopy_storageTier
--- PASS: TestAccEC2EBSSnapshotCopy_disappears (70.64s)
=== CONT  TestAccEC2EBSSnapshotCopy_withKMS
--- PASS: TestAccEC2EBSSnapshotCopy_withKMS (69.57s)
--- PASS: TestAccEC2EBSSnapshotCopy_storageTier (145.05s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	341.614s

@ewbankkit
Copy link
Contributor

@ahorner Thanks for the contribution 🎉 👏.

@ewbankkit
Copy link
Contributor

% make providerlint golangci-lint
==> Checking source code with providerlint...
==> Checking source code with golangci-lint...

@ewbankkit ewbankkit merged commit b9bcbfb into hashicorp:main Jun 6, 2022
@github-actions github-actions bot added this to the v4.18.0 milestone Jun 6, 2022
@ahorner ahorner deleted the f-aws_ebs_snapshot_copy-add_timeouts_support branch June 8, 2022 17:45
@github-actions
Copy link

This functionality has been released in v4.18.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/ec2ebs Issues and PRs that pertain to the ec2ebs service. size/XS Managed by automation to categorize the size of a PR.
Projects
None yet
5 participants