r/aws_kinesis_firehose_delivery_stream add timeouts (#22645)

[ameddin73]

Description

AWS Kinesis Firehose can take longer than 20 minutes to create (or destroy), but 20 minutes is the default timeout hardcoded into the provider resource.

This PR adds a Timeout field to the resource with the same 20 minute default to allow a user to extend it and prevent timeout failures.

Relations

Closes #22645

References

Output from Acceptance Testing

ameddin73:main:

TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20 -run='TestAccFirehoseDeliveryStream'  -timeout 180m
--- PASS: TestAccFirehoseDeliveryStream_s3KinesisStreamSource (70.35s)
--- PASS: TestAccFirehoseDeliveryStream_missingProcessing (182.34s)
--- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (184.02s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (221.57s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (160.73s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (233.27s)
--- PASS: TestAccFirehoseDeliveryStreamDataSource_basic (239.10s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithTags (239.71s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (63.05s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (316.68s)
--- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (317.85s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyType (324.81s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyARN (330.02s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (330.43s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (330.56s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSE (359.44s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (360.38s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (365.69s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (204.42s)
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (277.38s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithPrefixes (204.02s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (343.27s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3Updates (303.37s)
--- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (239.52s)
--- PASS: TestAccFirehoseDeliveryStream_s3Updates (205.07s)
--- PASS: TestAccFirehoseDeliveryStream_disappears (238.48s)
--- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (241.22s)
--- PASS: TestAccFirehoseDeliveryStream_tagUpdates (451.38s)
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (432.00s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (331.42s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (384.12s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3basic (372.49s)
--- FAIL: TestAccFirehoseDeliveryStream_basic (344.18s)
--- FAIL: TestAccFirehoseDeliveryStream_redshiftUpdates (520.98s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (321.32s)
--- PASS: TestAccFirehoseDeliveryStream_s3basic (314.33s)
--- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1478.85s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1577.14s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1641.99s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2546.94s)

hashicorp:main:

TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20 -run='TestAccFirehoseDeliveryStream'  -timeout 180m
go: downloading github.com/hashicorp/aws-cloudformation-resource-schema-sdk-go v0.20.0
--- PASS: TestAccFirehoseDeliveryStream_s3KinesisStreamSource (71.13s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (73.18s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (269.74s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (275.95s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (284.96s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (289.14s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithTags (289.79s)
--- PASS: TestAccFirehoseDeliveryStream_s3Updates (309.15s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3basic (314.41s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (323.45s)
--- PASS: TestAccFirehoseDeliveryStreamDataSource_basic (324.21s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (325.82s)
--- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (328.37s)
--- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (340.77s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (342.33s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (352.84s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (360.27s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3Updates (292.65s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSE (400.34s)
--- PASS: TestAccFirehoseDeliveryStream_tagUpdates (511.94s)
--- FAIL: TestAccFirehoseDeliveryStream_redshiftUpdates (794.76s)
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (565.80s)
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (563.64s)
--- FAIL: TestAccFirehoseDeliveryStream_basic (577.56s)
--- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (542.75s)
--- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (433.40s)
--- PASS: TestAccFirehoseDeliveryStream_disappears (633.61s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithPrefixes (641.62s)
--- PASS: TestAccFirehoseDeliveryStream_missingProcessing (720.91s)
--- PASS: TestAccFirehoseDeliveryStream_s3basic (701.03s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (668.52s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (673.49s)
--- FAIL: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (676.86s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (707.89s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyType (722.44s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyARN (795.60s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1546.77s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1538.52s)
--- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1562.06s)
--- FAIL: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2456.31s)

Test error related to #19936 or #26964

  Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # aws_kinesis_firehose_delivery_stream.test will be updated in-place
          ~ resource "aws_kinesis_firehose_delivery_stream" "test" {
                id             = "arn:aws:firehose:us-west-2:205063297161:deliverystream/tf-acc-test-5495822259980168874"
                name           = "tf-acc-test-5495822259980168874"
                tags           = {}
                # (5 unchanged attributes hidden)
              ~ elasticsearch_configuration {
                    # (9 unchanged attributes hidden)
                  ~ processing_configuration {
                        # (1 unchanged attribute hidden)
                      ~ processors {
                            # (1 unchanged attribute hidden)
                          - parameters {
                              - parameter_name  = "BufferSizeInMBs" -> null
                              - parameter_value = "1" -> null
                            }
                            # (1 unchanged block hidden)
                        }
                    }
                    # (2 unchanged blocks hidden)
                }
                # (2 unchanged blocks hidden)
            }

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

• Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
• Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

[github-actions]

Thank you for your contribution! 🚀

Please note that the CHANGELOG.md file contents are handled by the maintainers during merge. This is to prevent pull request merge conflicts, especially for contributions which may not be merged immediately. Please see the Contributing Guide for additional pull request review items.

Remove any changes to the CHANGELOG.md file and commit them in this pull request to prevent delays with reviewing and potentially merging this pull request.

[github-actions]

Welcome @ameddin73 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

[jar-b]

LGTM 🎉

$ make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_' PKG=firehose
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20  -run=TestAccFirehoseDeliveryStream_ -timeout 180m
=== CONT  TestAccFirehoseDeliveryStream_s3KinesisStreamSource
--- PASS: TestAccFirehoseDeliveryStream_s3KinesisStreamSource (62.21s)
=== CONT  TestAccFirehoseDeliveryStream_httpEndpoint
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (64.84s)
=== CONT  TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (129.74s)
=== CONT  TestAccFirehoseDeliveryStream_s3basicWithSSE
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (140.44s)
=== CONT  TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyType
--- PASS: TestAccFirehoseDeliveryStream_s3Updates (148.45s)
=== CONT  TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyARN
--- PASS: TestAccFirehoseDeliveryStream_missingProcessing (149.86s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (155.08s)
=== CONT  TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate
--- PASS: TestAccFirehoseDeliveryStream_extendedS3basic (159.59s)
=== CONT  TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning
--- PASS: TestAccFirehoseDeliveryStream_basic (159.85s)
=== CONT  TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN
--- PASS: TestAccFirehoseDeliveryStream_extendedS3Updates (241.41s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty
--- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (245.63s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (181.82s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (246.78s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithTags (249.59s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (256.82s)
=== CONT  TestAccFirehoseDeliveryStream_s3basic
--- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (194.84s)
=== CONT  TestAccFirehoseDeliveryStream_s3basicWithPrefixes
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (118.85s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (121.71s)
=== CONT  TestAccFirehoseDeliveryStream_disappears
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (219.51s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (210.64s)
=== CONT  TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (225.02s)
--- PASS: TestAccFirehoseDeliveryStream_redshiftUpdates (415.68s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (189.22s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (194.30s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (197.29s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyType (348.72s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSE (360.63s)
--- PASS: TestAccFirehoseDeliveryStream_s3basic (238.50s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithPrefixes (243.05s)
--- PASS: TestAccFirehoseDeliveryStream_s3basicWithSSEAndKeyARN (393.39s)
--- PASS: TestAccFirehoseDeliveryStream_tagUpdates (560.47s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (200.77s)
--- PASS: TestAccFirehoseDeliveryStream_disappears (212.47s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (242.89s)
--- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate (465.65s)
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (282.80s)
--- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1488.10s)
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1525.78s)
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1620.81s)
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2441.45s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/firehose   2444.638s

[jar-b]

Thanks for your contribution, @ameddin73!

[github-actions]

This functionality has been released in v4.55.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.