Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider/aws: Run all AWSConfig acc tests sequentially #13658

Merged
merged 1 commit into from
Apr 18, 2017

Conversation

radeksimko
Copy link
Member

This is to address the acceptance test failures caused by the limitations of AWSConfig (i.e. only 1 recorder/delivery channel can exist in 1 account/region at a time).

The subtest-based approach has only one downside - that is TC wrapper's inability to read the test results from subtests, so all of AWSConfig is tracked as a single test with all related details (e.g. flakiness).

Test plan

make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSConfig'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/14 10:18:53 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSConfig -timeout 120m
=== RUN   TestAccAWSConfig
=== RUN   TestAccAWSConfig/Config
=== RUN   TestAccAWSConfig/Config/basic
=== RUN   TestAccAWSConfig/Config/ownerAws
=== RUN   TestAccAWSConfig/Config/customlambda
=== RUN   TestAccAWSConfig/Config/importAws
=== RUN   TestAccAWSConfig/Config/importLambda
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/basic
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/startEnabled
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/importBasic
=== RUN   TestAccAWSConfig/ConfigurationRecorder
=== RUN   TestAccAWSConfig/ConfigurationRecorder/basic
=== RUN   TestAccAWSConfig/ConfigurationRecorder/allParams
=== RUN   TestAccAWSConfig/ConfigurationRecorder/importBasic
=== RUN   TestAccAWSConfig/DeliveryChannel
=== RUN   TestAccAWSConfig/DeliveryChannel/basic
=== RUN   TestAccAWSConfig/DeliveryChannel/allParams
=== RUN   TestAccAWSConfig/DeliveryChannel/importBasic
--- PASS: TestAccAWSConfig (4950.43s)
    --- PASS: TestAccAWSConfig/Config (1614.95s)
        --- PASS: TestAccAWSConfig/Config/basic (242.24s)
        --- PASS: TestAccAWSConfig/Config/ownerAws (237.00s)
        --- PASS: TestAccAWSConfig/Config/customlambda (431.92s)
        --- PASS: TestAccAWSConfig/Config/importAws (262.96s)
        --- PASS: TestAccAWSConfig/Config/importLambda (440.83s)
    --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus (1392.67s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/basic (305.05s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/startEnabled (779.85s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/importBasic (307.77s)
    --- PASS: TestAccAWSConfig/ConfigurationRecorder (989.80s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/basic (322.94s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/allParams (326.97s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/importBasic (339.89s)
    --- PASS: TestAccAWSConfig/DeliveryChannel (953.02s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/basic (339.02s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/allParams (301.06s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/importBasic (312.93s)
PASS
ok  	github.com/hashicorp/terraform/builtin/providers/aws	4950.462s

Copy link
Contributor

@stack72 stack72 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just ran all the tests in TeamCity and they ran great :)

[TestAccAWSConfig] [Test Output]
=== RUN   TestAccAWSConfig
=== RUN   TestAccAWSConfig/Config
=== RUN   TestAccAWSConfig/Config/ownerAws
=== RUN   TestAccAWSConfig/Config/customlambda
=== RUN   TestAccAWSConfig/Config/importAws
=== RUN   TestAccAWSConfig/Config/importLambda
=== RUN   TestAccAWSConfig/Config/basic
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/startEnabled
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/importBasic
=== RUN   TestAccAWSConfig/ConfigurationRecorderStatus/basic
=== RUN   TestAccAWSConfig/ConfigurationRecorder
=== RUN   TestAccAWSConfig/ConfigurationRecorder/basic
=== RUN   TestAccAWSConfig/ConfigurationRecorder/allParams
=== RUN   TestAccAWSConfig/ConfigurationRecorder/importBasic
=== RUN   TestAccAWSConfig/DeliveryChannel
=== RUN   TestAccAWSConfig/DeliveryChannel/basic
=== RUN   TestAccAWSConfig/DeliveryChannel/allParams
=== RUN   TestAccAWSConfig/DeliveryChannel/importBasic
--- PASS: TestAccAWSConfig (727.47s)
    --- PASS: TestAccAWSConfig/Config (468.55s)
        --- PASS: TestAccAWSConfig/Config/ownerAws (88.17s)
        --- PASS: TestAccAWSConfig/Config/customlambda (102.52s)
        --- PASS: TestAccAWSConfig/Config/importAws (88.62s)
        --- PASS: TestAccAWSConfig/Config/importLambda (101.44s)
        --- PASS: TestAccAWSConfig/Config/basic (87.80s)
    --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus (96.28s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/startEnabled (39.37s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/importBasic (29.71s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorderStatus/basic (27.20s)
    --- PASS: TestAccAWSConfig/ConfigurationRecorder (79.89s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/basic (25.27s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/allParams (27.22s)
        --- PASS: TestAccAWSConfig/ConfigurationRecorder/importBasic (27.40s)
    --- PASS: TestAccAWSConfig/DeliveryChannel (82.75s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/basic (26.70s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/allParams (28.08s)
        --- PASS: TestAccAWSConfig/DeliveryChannel/importBasic (27.96s)
PASS

@stack72 stack72 merged commit 1b841e7 into master Apr 18, 2017
@stack72 stack72 deleted the b-awsconfig-acc-tests branch April 18, 2017 15:00
@ghost
Copy link

ghost commented Apr 13, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants