Add auto tune options to aws_elasticsearch_domain

[zhelding]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Adds the ability to set auto_tune_options on resource type aws_elasticsearch_domain. Includes the ability to set maintenance_schedule list and rollback options.

Closes #18421

[zhelding]

Output from acceptance tests:

[I] ➜ make testacc TESTARGS='-run=TestAccElasticsearchDomain\(DataSource\|_AutoTuneOptions\)' PKG_NAME=internal/service/elasticsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run=TestAccElasticsearchDomain\(DataSource\|_AutoTuneOptions\) -timeout 180m
=== RUN   TestAccElasticsearchDomainDataSource_Data_basic
=== PAUSE TestAccElasticsearchDomainDataSource_Data_basic
=== RUN   TestAccElasticsearchDomainDataSource_Data_advanced
=== PAUSE TestAccElasticsearchDomainDataSource_Data_advanced
=== RUN   TestAccElasticsearchDomain_AutoTuneOptions
=== PAUSE TestAccElasticsearchDomain_AutoTuneOptions
=== CONT  TestAccElasticsearchDomainDataSource_Data_basic
=== CONT  TestAccElasticsearchDomain_AutoTuneOptions
=== CONT  TestAccElasticsearchDomainDataSource_Data_advanced
--- PASS: TestAccElasticsearchDomainDataSource_Data_basic (1038.42s)
--- PASS: TestAccElasticsearchDomain_AutoTuneOptions (1209.40s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_advanced (1413.61s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/elasticsearch      1413.643s

[zhelding]

Acceptance tests still passing with further edits:

[I] ➜ make testacc TESTARGS='-run=TestAccElasticsearchDomain\(DataSource\|_AutoTuneOptions\)' PKG_NAME=internal/service/elasticsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run=TestAccElasticsearchDomain\(DataSource\|_AutoTuneOptions\) -timeout 180m
=== RUN   TestAccElasticsearchDomainDataSource_Data_basic
=== PAUSE TestAccElasticsearchDomainDataSource_Data_basic
=== RUN   TestAccElasticsearchDomainDataSource_Data_advanced
=== PAUSE TestAccElasticsearchDomainDataSource_Data_advanced
=== RUN   TestAccElasticsearchDomain_AutoTuneOptions
=== PAUSE TestAccElasticsearchDomain_AutoTuneOptions
=== CONT  TestAccElasticsearchDomainDataSource_Data_basic
=== CONT  TestAccElasticsearchDomain_AutoTuneOptions
=== CONT  TestAccElasticsearchDomainDataSource_Data_advanced
--- PASS: TestAccElasticsearchDomain_AutoTuneOptions (1220.27s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_basic (1221.73s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_advanced (1534.79s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/elasticsearch      1534.822s

[ewbankkit]

When I run the full set of aws_elasticsearch_domain tests I get a number of failure like:

=== CONT  TestAccElasticsearchDomain_basic
    domain_test.go:30: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:
        
        
        Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place
        
        Terraform will perform the following actions:
        
          # aws_elasticsearch_domain.test will be updated in-place
          ~ resource "aws_elasticsearch_domain" "test" {
                id                    = "arn:aws:es:us-west-2:123456789012:domain/tf-test-2139854371124166734"
                # (8 unchanged attributes hidden)
        
        
              - auto_tune_options {
                  - desired_state       = "DISABLED" -> null
                  - rollback_on_disable = "NO_ROLLBACK" -> null
                }
        
        
        
        
        
        
        
                # (8 unchanged blocks hidden)
            }
        
        Plan: 0 to add, 1 to change, 0 to destroy.

which is for the configuration

resource "aws_elasticsearch_domain" "test" {
  domain_name = "tf-test-%d"

  ebs_options {
    ebs_enabled = true
    volume_size = 10
  }
}

This suggests you should mark auto_tune_options with Computed: true, similar to domain_endpoint_options.

[zhelding]

This PR requires the upcoming v2.9.0 of terraform-plugin-sdk and so will be merged upon its release.

On v2.8.1 of terraform-plugin-sdk, this PR reports a diff in auto_tune_options where none exists -- as it contains a Set nested in a Map (hashicorp/terraform-plugin-sdk#617).

Output from acceptance tests using v2.9.0:

[I] ➜ make testacc PKG_NAME=internal/service/elasticsearch TESTARGS='-run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_ -timeout 180m
--- PASS: TestAccElasticsearchDomain_WithVolumeType_missing (909.97s)
--- PASS: TestAccElasticsearchDomain_tags (936.00s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs (1302.85s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_basic (1361.12s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_auditLogs (1388.72s)
--- PASS: TestAccElasticsearchDomain_v23 (1394.33s)
--- PASS: TestAccElasticsearchDomain_EncryptAtRestDefault_key (1481.30s)
--- PASS: TestAccElasticsearchDomain_policy (1577.42s)
--- PASS: TestAccElasticsearchDomain_nodeToNodeEncryption (1593.77s)
--- PASS: TestAccElasticsearchDomain_EncryptAtRestSpecify_key (1715.70s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs (1906.24s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB (1938.90s)
--- PASS: TestAccElasticsearchDomain_AutoTuneOptions (1180.38s)
--- PASS: TestAccElasticsearchDomain_duplicate (640.06s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove (2368.70s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs (2486.82s)
--- PASS: TestAccElasticsearchDomain_vpc (1196.54s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled (2623.01s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_iam (2753.14s)
--- PASS: TestAccElasticsearchDomain_basic (1337.92s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsUpdate (3367.45s)
--- PASS: TestAccElasticsearchDomain_customEndpoint (1725.36s)
--- PASS: TestAccElasticsearchDomain_VPC_update (2411.73s)
--- PASS: TestAccElasticsearchDomain_requireHTTPS (2236.58s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_advanced (1982.38s)
--- PASS: TestAccElasticsearchDomain_Update_version (4269.70s)
--- PASS: TestAccElasticsearchDomain_internetToVPCEndpoint (3582.78s)
--- PASS: TestAccElasticsearchDomain_withDedicatedMaster (3312.26s)
--- PASS: TestAccElasticsearchDomain_warm (4631.32s)
--- PASS: TestAccElasticsearchDomain_Cluster_zoneAwareness (6153.89s)
--- FAIL: TestAccElasticsearchDomain_update (2279.37s) -- unrelated (advanced options drift)
--- FAIL: TestAccElasticsearchDomain_complex (1208.41s) -- unrelated (advanced options drift)
--- FAIL: TestAccElasticsearchDomain_UpdateVolume_type (3715.16s) -- unrelated (advanced options drift)

Remaining test failures are due to separate issue of perpetual advanced_options drift: #21318.

[pawelmitka]

https://github.com/hashicorp/terraform-plugin-sdk/releases/tag/v2.9.0 has been released 12 days ago. What are the plans to release this feature?

[zhelding]

terraform-plugin-sdk v2.9.0 has been adopted by main: #21911.

Acceptance tests are passing on latest commit after git merge main:

[I] ➜ make testacc PKG_NAME=internal/service/elasticsearch TESTARGS='-run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_ -timeout 180m
--- PASS: TestAccElasticsearchDomain_duplicate (759.81s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs (1163.86s)
--- PASS: TestAccElasticsearchDomain_EncryptAtRestSpecify_key (1184.85s)
--- PASS: TestAccElasticsearchDomain_EncryptAtRestDefault_key (1289.02s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled (1323.64s)
--- PASS: TestAccElasticsearchDomain_nodeToNodeEncryption (1528.48s)
--- PASS: TestAccElasticsearchDomain_WithVolumeType_missing (1609.05s)
--- PASS: TestAccElasticsearchDomain_tags (1771.73s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs (2163.32s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_iam (1073.27s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsUpdate (2568.00s)
--- PASS: TestAccElasticsearchDomain_policy (2581.86s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB (1430.69s)
--- PASS: TestAccElasticsearchDomain_basic (1020.83s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_auditLogs (2639.78s)
--- PASS: TestAccElasticsearchDomain_requireHTTPS (1516.92s)
--- PASS: TestAccElasticsearchDomain_AutoTuneOptions (1561.48s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs (2896.84s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_basic (2963.65s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove (2549.92s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_advanced (1637.66s)
--- PASS: TestAccElasticsearchDomain_withDedicatedMaster (3621.50s)
--- PASS: TestAccElasticsearchDomain_v23 (1117.84s)
--- PASS: TestAccElasticsearchDomain_vpc (1969.72s)
--- PASS: TestAccElasticsearchDomain_Update_version (3829.86s)
--- PASS: TestAccElasticsearchDomain_internetToVPCEndpoint (2758.42s)
--- PASS: TestAccElasticsearchDomain_VPC_update (2439.40s)
--- PASS: TestAccElasticsearchDomain_customEndpoint (3352.89s)
--- PASS: TestAccElasticsearchDomain_warm (5041.44s)
--- PASS: TestAccElasticsearchDomain_Cluster_zoneAwareness (5909.82s)
--- FAIL: TestAccElasticsearchDomain_update (1185.08s) -- unrelated (advanced options drift)
--- FAIL: TestAccElasticsearchDomain_complex (1273.97s) -- unrelated (advanced options drift)
--- FAIL: TestAccElasticsearchDomain_UpdateVolume_type (1791.16s) -- unrelated (advanced options drift)

Remaining test failures still attributable to separate issue of perpetual advanced_options drift: #21318.

Merge pending final review.

[ewbankkit]

LGTM 🚀.

% make testacc PKG_NAME=internal/service/elasticsearch TESTARGS='-run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run=TestAccElasticsearchDomainDataSource_\|TestAccElasticsearchDomain_ -timeout 180m
=== RUN   TestAccElasticsearchDomainDataSource_Data_basic
=== PAUSE TestAccElasticsearchDomainDataSource_Data_basic
=== RUN   TestAccElasticsearchDomainDataSource_Data_advanced
=== PAUSE TestAccElasticsearchDomainDataSource_Data_advanced
=== RUN   TestAccElasticsearchDomain_basic
=== PAUSE TestAccElasticsearchDomain_basic
=== RUN   TestAccElasticsearchDomain_requireHTTPS
=== PAUSE TestAccElasticsearchDomain_requireHTTPS
=== RUN   TestAccElasticsearchDomain_customEndpoint
=== PAUSE TestAccElasticsearchDomain_customEndpoint
=== RUN   TestAccElasticsearchDomain_Cluster_zoneAwareness
=== PAUSE TestAccElasticsearchDomain_Cluster_zoneAwareness
=== RUN   TestAccElasticsearchDomain_warm
=== PAUSE TestAccElasticsearchDomain_warm
=== RUN   TestAccElasticsearchDomain_withDedicatedMaster
=== PAUSE TestAccElasticsearchDomain_withDedicatedMaster
=== RUN   TestAccElasticsearchDomain_duplicate
=== PAUSE TestAccElasticsearchDomain_duplicate
=== RUN   TestAccElasticsearchDomain_v23
=== PAUSE TestAccElasticsearchDomain_v23
=== RUN   TestAccElasticsearchDomain_complex
=== PAUSE TestAccElasticsearchDomain_complex
=== RUN   TestAccElasticsearchDomain_vpc
=== PAUSE TestAccElasticsearchDomain_vpc
=== RUN   TestAccElasticsearchDomain_VPC_update
=== PAUSE TestAccElasticsearchDomain_VPC_update
=== RUN   TestAccElasticsearchDomain_internetToVPCEndpoint
=== PAUSE TestAccElasticsearchDomain_internetToVPCEndpoint
=== RUN   TestAccElasticsearchDomain_AutoTuneOptions
=== PAUSE TestAccElasticsearchDomain_AutoTuneOptions
=== RUN   TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB
=== PAUSE TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB
=== RUN   TestAccElasticsearchDomain_AdvancedSecurityOptions_iam
=== PAUSE TestAccElasticsearchDomain_AdvancedSecurityOptions_iam
=== RUN   TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled
=== PAUSE TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled
=== RUN   TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs
=== PAUSE TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs
=== RUN   TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs
=== PAUSE TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs
=== RUN   TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs
=== PAUSE TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs
=== RUN   TestAccElasticsearchDomain_LogPublishingOptions_auditLogs
=== PAUSE TestAccElasticsearchDomain_LogPublishingOptions_auditLogs
=== RUN   TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove
=== PAUSE TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove
=== RUN   TestAccElasticsearchDomain_cognitoOptionsUpdate
=== PAUSE TestAccElasticsearchDomain_cognitoOptionsUpdate
=== RUN   TestAccElasticsearchDomain_policy
=== PAUSE TestAccElasticsearchDomain_policy
=== RUN   TestAccElasticsearchDomain_EncryptAtRestDefault_key
=== PAUSE TestAccElasticsearchDomain_EncryptAtRestDefault_key
=== RUN   TestAccElasticsearchDomain_EncryptAtRestSpecify_key
=== PAUSE TestAccElasticsearchDomain_EncryptAtRestSpecify_key
=== RUN   TestAccElasticsearchDomain_nodeToNodeEncryption
=== PAUSE TestAccElasticsearchDomain_nodeToNodeEncryption
=== RUN   TestAccElasticsearchDomain_tags
=== PAUSE TestAccElasticsearchDomain_tags
=== RUN   TestAccElasticsearchDomain_update
=== PAUSE TestAccElasticsearchDomain_update
=== RUN   TestAccElasticsearchDomain_UpdateVolume_type
=== PAUSE TestAccElasticsearchDomain_UpdateVolume_type
=== RUN   TestAccElasticsearchDomain_WithVolumeType_missing
=== PAUSE TestAccElasticsearchDomain_WithVolumeType_missing
=== RUN   TestAccElasticsearchDomain_Update_version
=== PAUSE TestAccElasticsearchDomain_Update_version
=== CONT  TestAccElasticsearchDomainDataSource_Data_basic
=== CONT  TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled
=== CONT  TestAccElasticsearchDomain_Update_version
=== CONT  TestAccElasticsearchDomain_WithVolumeType_missing
=== CONT  TestAccElasticsearchDomain_UpdateVolume_type
=== CONT  TestAccElasticsearchDomain_update
=== CONT  TestAccElasticsearchDomain_tags
=== CONT  TestAccElasticsearchDomain_nodeToNodeEncryption
=== CONT  TestAccElasticsearchDomain_EncryptAtRestSpecify_key
=== CONT  TestAccElasticsearchDomain_EncryptAtRestDefault_key
=== CONT  TestAccElasticsearchDomain_policy
=== CONT  TestAccElasticsearchDomain_cognitoOptionsUpdate
=== CONT  TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove
=== CONT  TestAccElasticsearchDomain_LogPublishingOptions_auditLogs
=== CONT  TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs
=== CONT  TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs
=== CONT  TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs
=== CONT  TestAccElasticsearchDomain_v23
=== CONT  TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB
=== CONT  TestAccElasticsearchDomain_AutoTuneOptions
=== CONT  TestAccElasticsearchDomain_update
=== CONT  TestAccElasticsearchDomain_UpdateVolume_type
--- PASS: TestAccElasticsearchDomain_nodeToNodeEncryption (990.63s)
=== CONT  TestAccElasticsearchDomain_internetToVPCEndpoint
--- FAIL: TestAccElasticsearchDomain_update (1044.29s)
=== CONT  TestAccElasticsearchDomain_VPC_update
--- FAIL: TestAccElasticsearchDomain_UpdateVolume_type (1109.43s)
=== CONT  TestAccElasticsearchDomain_vpc
--- PASS: TestAccElasticsearchDomain_WithVolumeType_missing (1165.91s)
=== CONT  TestAccElasticsearchDomain_complex
--- PASS: TestAccElasticsearchDomain_v23 (1183.04s)
=== CONT  TestAccElasticsearchDomain_Cluster_zoneAwareness
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_indexSlowLogs (1273.60s)
=== CONT  TestAccElasticsearchDomain_duplicate
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_esApplicationLogs (1351.68s)
=== CONT  TestAccElasticsearchDomain_withDedicatedMaster
--- PASS: TestAccElasticsearchDomain_policy (1370.77s)
=== CONT  TestAccElasticsearchDomain_warm
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_searchSlowLogs (1429.45s)
=== CONT  TestAccElasticsearchDomain_requireHTTPS
--- PASS: TestAccElasticsearchDomainDataSource_Data_basic (1586.49s)
=== CONT  TestAccElasticsearchDomain_customEndpoint
--- PASS: TestAccElasticsearchDomain_EncryptAtRestSpecify_key (1645.83s)
=== CONT  TestAccElasticsearchDomain_basic
--- PASS: TestAccElasticsearchDomain_AutoTuneOptions (1833.25s)
=== CONT  TestAccElasticsearchDomainDataSource_Data_advanced
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_userDB (1879.31s)
=== CONT  TestAccElasticsearchDomain_complex
--- PASS: TestAccElasticsearchDomain_duplicate (886.57s)
--- FAIL: TestAccElasticsearchDomain_complex (1037.70s)
--- PASS: TestAccElasticsearchDomain_EncryptAtRestDefault_key (2436.59s)
--- PASS: TestAccElasticsearchDomain_AdvancedSecurityOptions_disabled (2464.77s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsUpdate (2478.06s)
--- PASS: TestAccElasticsearchDomain_vpc (1415.23s)
--- PASS: TestAccElasticsearchDomain_tags (2646.90s)
--- PASS: TestAccElasticsearchDomain_cognitoOptionsCreateAndRemove (2813.68s)
--- PASS: TestAccElasticsearchDomain_LogPublishingOptions_auditLogs (2935.44s)
--- PASS: TestAccElasticsearchDomain_requireHTTPS (1591.62s)
--- PASS: TestAccElasticsearchDomain_basic (1436.70s)
--- PASS: TestAccElasticsearchDomain_customEndpoint (1724.33s)
--- PASS: TestAccElasticsearchDomainDataSource_Data_advanced (1540.55s)
--- PASS: TestAccElasticsearchDomain_VPC_update (2615.26s)
--- PASS: TestAccElasticsearchDomain_internetToVPCEndpoint (2905.05s)
--- PASS: TestAccElasticsearchDomain_withDedicatedMaster (3419.59s)
--- PASS: TestAccElasticsearchDomain_Update_version (5242.57s)
--- PASS: TestAccElasticsearchDomain_warm (4457.38s)
--- PASS: TestAccElasticsearchDomain_Cluster_zoneAwareness (5890.88s)

The failures are due to #21318.

[zhelding]

Identical acceptance tests results with latest commit 8b369e6 making minor logic adjustment suggested by @ewbankkit.

Merging to main!

@pawelmitka: this feature will be part of our next regular release (v3.69.0) on Thursday, December 9th.

[github-actions]

This functionality has been released in v3.69.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.