test/datasource/aws_elasticsearch_domain: Remove service role resource creation

[nywilken]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Release note for CHANGELOG:

none

Acceptance test before change:

--- FAIL: TestAccAWSDataElasticsearchDomain_advanced (18.37s)
    testing.go:569: Step 0 error: errors during apply:

        Error: Error creating service-linked role with name es.amazonaws.com: InvalidInput: Service role name AWSServiceRoleForAmazonElasticsearchService has been taken in this account, please try a different suffix.
        	status code: 400, request id: e62f1d43-d04c-11e9-ae4f-bb716bff761d

          on /opt/teamcity-agent/temp/buildTmp/tf-test399323254/main.tf line 70:
          (source code not available)

Acceptance test after change:

--- PASS: TestAccAWSDataElasticsearchDomain_advanced (18.37s)

[bflad]

Hey @nywilken 👋 Thank you for fixing this! Certainly an annoyance with the acceptance testing for the ElasticSearch resources since each account must have the es.{DNS_SUFFIX} IAM Service Linked Role created once before attempting to create an Elasticsearch Domain that requires it (unlike other AWS services that automatically create the IAM SLR for you). For example, in an account without the IAM SLR:

--- FAIL: TestAccAWSDataElasticsearchDomain_advanced (159.86s)
    testing.go:569: Step 0 error: errors during apply:

        Error: ValidationException: Before you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC.

Given this requirement, we should ensure the acceptance testing for this data source (and likely the other aws_elasticsearch_domain resource tests too) include a PreCheck function for the existence of the es IAM SLR in the AWS account before running the test, e.g.

In aws/resource_aws_elasticsearch_domain_test.go (we typically include PreCheck functions in the resource testing files):

func testAccPreCheckIamServiceLinkedRoleEs(t *testing.T) {
	conn := testAccProvider.Meta().(*AWSClient).iamconn
	dnsSuffix := testAccProvider.Meta().(*AWSClient).dnsSuffix

	input := &iam.ListRolesInput{
		PathPrefix: aws.String("/aws-service-role/es."),
	}

	var role *iam.Role
	err := conn.ListRolesPages(input, func(page *iam.ListRolesOutput, lastPage bool) bool {
		for _, r := range page.Roles {
			if strings.HasPrefix(aws.StringValue(r.Path), "/aws-service-role/es.") {
				role = r
			}
		}

		return !lastPage
	})

	if testAccPreCheckSkipError(err) {
		t.Skipf("skipping acceptance testing: %s", err)
	}

	if err != nil {
		t.Fatalf("unexpected PreCheck error: %s", err)
	}

	if role == nil {
		t.Fatalf("missing IAM Service Linked Role (es.%s), please create it in the AWS account and retry", dnsSuffix)
	}
}

And for each relevant test:

PreCheck:  func() { testAccPreCheck(t); testAccPreCheckIamServiceLinkedRoleEs(t) },

This will allow the testing to fail fast at the beginning, for example:

--- FAIL: TestAccAWSDataElasticsearchDomain_advanced (9.19s)
    resource_aws_elasticsearch_domain_test.go:827: missing IAM Service Linked Role (es.amazonaws.com), please create it in the AWS account and retry

When the IAM SLR is present, testing proceeds as normal. 👍

That said, it looks like these particular test configurations also have some other issues --

In AWS Commercial:

--- FAIL: TestAccAWSDataElasticsearchDomain_advanced (123.93s)
    testing.go:569: Step 0 error: errors during apply:

        Error: ValidationException: Each subnet must be in its own Availability Zone.

This is because each the aws_subnet resource declarations doesn't include an availablility_zone argument, which can be coupled with the aws_availability_zones data source to ensure its region-agnostic and only available AZs, e.g.

data "aws_availability_zones" "available" {
  state = "available"
}

# ...

resource "aws_subnet" "test" {
  availability_zone = "${data.aws_availability_zones.available.names[0]}"
  cidr_block        = "10.0.0.0/24"
  vpc_id            = "${aws_vpc.test.id}"
}

resource "aws_subnet" "test2" {
  availability_zone = "${data.aws_availability_zones.available.names[1]}"
  cidr_block        = "10.0.1.0/24"
  vpc_id            = "${aws_vpc.test.id}"
}

In AWS GovCloud (US):

--- FAIL: TestAccAWSDataElasticsearchDomain_advanced (13.42s)
    testing.go:569: Step 0 error: errors during refresh:

        Error: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.

The test configurations are using a hardcoded provider configuration set to region = us-east-1. This testing should not require that specific region and the provider configuration in the test configuration can be removed. 😄

As for managing the actual IAM SLRs, I'll reach out to you directly about managing these via our acceptance test account Terraform configurations so they are accounted for in each of those accounts. 👍

[bflad]

Pulling in fix for this data source test, will handle #10035 (review) separately, shortly.

[ghost]

This has been released in version 2.32.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!