[#19966] - aws_elasticsearch_domain_saml_options subject_key default differs from AWS

[jc-asdf]

Fixed by making the subject_key's default value be "" - an empty string - to match the default on the AWS side.

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #19966

Output from acceptance testing:

 AWS_REGION=us-east-1 AWS_PROFILE=sandbox make testacc TESTARGS='-run=TestAccAWSElasticSearchDomainSAMLOptions'

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSElasticSearchDomainSAMLOptions -timeout 180m
=== RUN   TestAccAWSElasticSearchDomainSAMLOptions_basic
=== PAUSE TestAccAWSElasticSearchDomainSAMLOptions_basic
=== RUN   TestAccAWSElasticSearchDomainSAMLOptions_disappears
=== PAUSE TestAccAWSElasticSearchDomainSAMLOptions_disappears
=== RUN   TestAccAWSElasticSearchDomainSAMLOptions_disappears_Domain
=== PAUSE TestAccAWSElasticSearchDomainSAMLOptions_disappears_Domain
=== RUN   TestAccAWSElasticSearchDomainSAMLOptions_Update
=== PAUSE TestAccAWSElasticSearchDomainSAMLOptions_Update
=== RUN   TestAccAWSElasticSearchDomainSAMLOptions_Disabled
=== PAUSE TestAccAWSElasticSearchDomainSAMLOptions_Disabled
=== CONT  TestAccAWSElasticSearchDomainSAMLOptions_basic
=== CONT  TestAccAWSElasticSearchDomainSAMLOptions_Update
=== CONT  TestAccAWSElasticSearchDomainSAMLOptions_Disabled
=== CONT  TestAccAWSElasticSearchDomainSAMLOptions_disappears_Domain
=== CONT  TestAccAWSElasticSearchDomainSAMLOptions_disappears
--- PASS: TestAccAWSElasticSearchDomainSAMLOptions_disappears_Domain (1263.67s)
--- PASS: TestAccAWSElasticSearchDomainSAMLOptions_basic (1323.29s)
--- PASS: TestAccAWSElasticSearchDomainSAMLOptions_Update (1380.93s)
--- PASS: TestAccAWSElasticSearchDomainSAMLOptions_disappears (1580.45s)
--- PASS: TestAccAWSElasticSearchDomainSAMLOptions_Disabled (1713.77s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	1716.013s

[jc-asdf]

To verify the AWS-side default, I created a working Elasticsearch cluster via the AWS UI, enabling SAML with our Okta integration, leaving the subject key field empty. I got it working from end-to-end - from signing into SSO (Okta), through to the Kibana interface.

I verified that the SAML assertion had a NameID element in the Subject section - it did. I used the CLI to do a describe:

aws es describe-elasticsearch-domain --domain-name=audit-app-staging --profile=sandbox --region=us-east-1

{
    "DomainStatus": {
        "DomainId": "XXXXXXXX/terraform-test-saml",
        "DomainName": "terraform-test-saml",
        "ARN": "arn:aws:es:us-east-1:XXXXXXXX:domain/terraform-test-saml",
        "Created": true,
        "Deleted": false,

... <cut out the bulk> ...

        "AdvancedSecurityOptions": {
            "Enabled": true,
            "InternalUserDatabaseEnabled": false,
            "SAMLOptions": {
                "Enabled": true,
                "Idp": {
                    ... <cut out the XML idP metadata> ...
                },
                "SubjectKey": "",
                "RolesKey": "roles",
                "SessionTimeoutMinutes": 60
            }
        },
        "AutoTuneOptions": {
            "State": "ENABLE_IN_PROGRESS"
        }
    }
}

From here I could see that SubjectKey was in fact an empty string. I replicated the original bug by setting the SubjectKey to NameID (which, incidentally, can NOT be done in the AWS UI - it says that this is an invalid option; and so one must use the API to set it to this value). Once it was changed, I received a 500 error from Kibana; unfortunately I could not get useful error logs from it. The errors went away once this was changed back to a "", whether in the UI or via the API.

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[jc-asdf]

The changes in paths from #20000 have been merged in; this PR is compliant with the updated style / layout.

[Jacendb]

I'm seeing the same issue when creating a new ES domain. Does subject_key need a default value? In the AWS console I currently have to remove NameID and leave the Subject key box empty for SSO/Okta to work. I'm wondering if not setting anything in default will leave the field untouched/undefined on the AWS side so it works.

[YakDriver]

I agree that normally, changing the default value is a breaking change. However, in this case, the old and new default values are equivalent. Thus, I don't see this as a breaking change.

[YakDriver]

Thank you 🎉

% make testacc TESTS=TestAccElasticsearchDomainSAMLOptions PKG=elasticsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/elasticsearch/... -v -count 1 -parallel 20 -run='TestAccElasticsearchDomainSAMLOptions'  -timeout 180m
--- PASS: TestAccElasticsearchDomainSAMLOptions_disappears (1178.57s)
--- PASS: TestAccElasticsearchDomainSAMLOptions_Update (1202.84s)
--- PASS: TestAccElasticsearchDomainSAMLOptions_disappears_Domain (1268.98s)
--- PASS: TestAccElasticsearchDomainSAMLOptions_Disabled (1373.32s)
--- PASS: TestAccElasticsearchDomainSAMLOptions_basic (1450.98s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/elasticsearch	1452.310s

[github-actions]

This functionality has been released in v4.9.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.