resource/aws_emrcontainers_virtual_cluster: new resource

[shuheiktgw]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates #16717

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccXXX'

...

[DrFaust92]

Not needed as TF always has a top level id attribute

[shuheiktgw]

Didn't know that! Thank you!

[DrFaust92]

Lets add validation to name based on docs: https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_CreateVirtualCluster.html#API_CreateVirtualCluster_RequestSyntax

Len is between 1 and 64
and must conform to the following regex: [\.\-_/#A-Za-z0-9]+

[DrFaust92]

Missing SetId to the cluster id

[shuheiktgw]

Hmm, while writing acc tests, I noticed those two steps are not possible through the current terraform-aws-provider:

• https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-cluster-access.html
• https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-enable-IAM.html

1. We cannot create Kubernetes resources from terraform-aws-provider
2. terraform-aws-provider still hasn't support EKS addons, which is necessary for IAM Roles for service accounts (Support Managing EKS Cluster Add-Ons #16542)

The second problem will be solved eventually but I'm not sure about how I can solve the first one... @DrFaust92 Do you have any ideas? 🙏

[DrFaust92]

Hey @shuheiktgw, do acc tests fail without those prerequisites? I don’t how they block creating the resource and using additional resources (and providers) to add access outside the scope of the tests

[shuheiktgw]

@DrFaust92 Yes without them, the acc test fails with the following error:

make testacc TESTARGS='-run=TestAccAwsEMRContainersVirtualCluster_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAwsEMRContainersVirtualCluster_basic -timeout 120m
=== RUN   TestAccAwsEMRContainersVirtualCluster_basic
=== PAUSE TestAccAwsEMRContainersVirtualCluster_basic
=== CONT  TestAccAwsEMRContainersVirtualCluster_basic
    resource_aws_emrcontainers_virtual_cluster_test.go:65: Step 1/2 error: Error running apply:
        Error: error creating EMR containers virtual cluster: ValidationException: Unauthorized to perform read namespace on default

I've looked into eksctl's PR to enable EMR containers, and they seem to create Kubernetes resources...

• Support enabling access for EMR containers eksctl-io/eksctl#2913

[shuheiktgw]

@bflad @ewbankkit Do you people know how I can write an acc test that seemingly requires provisioning Kubernetes resources?

[bflad]

@shuheiktgw if it does not work by just including provider "kubernetes" {} in the configuration, it may be necessary to also use helper/resource.TestCase type ExternalProviders field, however we do not currently have any examples of multiple provider testing in this codebase anymore. Attempting to configure the Kubernetes provider using values from other resources may also require two TestStep.

The maintainers are very behind on the large backlog of existing pull requests, so unfortunately we cannot provide more guidance ourselves or guarantee when this will be reviewed.

[shuheiktgw]

Thank you @bflad! With that information, I think I can start writing the acc tests 👍

[anggao]

@shuheiktgw do you have any updates on this PR :) looking forward to use it

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[ewbankkit]

Superseded by #20003.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.