provider/aws: Use mutex & retry for WAF change operations

[radeksimko]

This is to address the following long term WAF test failures:

=== RUN   TestAccAWSWafByteMatchSet_changeNameForceNew
--- FAIL: TestAccAWSWafByteMatchSet_changeNameForceNew (17.11s)
    testing.go:280: Step 1 error: Error applying: 1 error(s) occurred:
        
        * aws_waf_byte_match_set.byte_set (destroy): 1 error(s) occurred:
        
        * aws_waf_byte_match_set.byte_set: [ERROR] Error deleting ByteMatchSet: [ERROR] Error updating ByteMatchSet: WAFStaleDataException: The input token is no longer current.
            status code: 400, request id: 2ea7dc97-20e2-11e7-b79e-d1acbf1c3f7f
    testing.go:344: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.
        

Closes #10335

Test plan

=== RUN   TestAccAWSWafByteMatchSet_disappears
--- PASS: TestAccAWSWafByteMatchSet_disappears (29.46s)
=== RUN   TestAccAWSWafIPSet_basic
--- PASS: TestAccAWSWafIPSet_basic (44.91s)
=== RUN   TestAccAWSWafIPSet_disappears
--- PASS: TestAccAWSWafIPSet_disappears (65.57s)
=== RUN   TestAccAWSWafSqlInjectionMatchSet_basic
--- PASS: TestAccAWSWafSqlInjectionMatchSet_basic (83.43s)
=== RUN   TestAccAWSWafSqlInjectionMatchSet_disappears
--- PASS: TestAccAWSWafSqlInjectionMatchSet_disappears (111.31s)
=== RUN   TestAccAWSWafXssMatchSet_disappears
--- PASS: TestAccAWSWafXssMatchSet_disappears (67.50s)
=== RUN   TestAccAWSWafSizeConstraintSet_changeNameForceNew
--- PASS: TestAccAWSWafSizeConstraintSet_changeNameForceNew (115.84s)
=== RUN   TestAccAWSWafSizeConstraintSet_basic
--- PASS: TestAccAWSWafSizeConstraintSet_basic (138.74s)
=== RUN   TestAccAWSWafSizeConstraintSet_disappears
--- PASS: TestAccAWSWafSizeConstraintSet_disappears (160.11s)
=== RUN   TestAccAWSWafSqlInjectionMatchSet_changeNameForceNew
--- PASS: TestAccAWSWafSqlInjectionMatchSet_changeNameForceNew (171.16s)
=== RUN   TestAccAWSWafRule_disappears
--- PASS: TestAccAWSWafRule_disappears (199.93s)
=== RUN   TestAccAWSWafByteMatchSet_basic
--- PASS: TestAccAWSWafByteMatchSet_basic (209.15s)
=== RUN   TestAccAWSWafByteMatchSet_changeNameForceNew
--- PASS: TestAccAWSWafByteMatchSet_changeNameForceNew (228.33s)
=== RUN   TestAccAWSWafRule_changeNameForceNew
--- PASS: TestAccAWSWafRule_changeNameForceNew (271.58s)
=== RUN   TestAccAWSWafWebAcl_changeDefaultAction
--- PASS: TestAccAWSWafWebAcl_changeDefaultAction (282.30s)
=== RUN   TestAccAWSWafIPSet_changeNameForceNew
--- PASS: TestAccAWSWafIPSet_changeNameForceNew (293.37s)
=== RUN   TestAccAWSWafXssMatchSet_changeNameForceNew
--- PASS: TestAccAWSWafXssMatchSet_changeNameForceNew (268.99s)
=== RUN   TestAccAWSWafXssMatchSet_basic
--- PASS: TestAccAWSWafXssMatchSet_basic (300.87s)
=== RUN   TestAccAWSWafWebAcl_basic
--- PASS: TestAccAWSWafWebAcl_basic (309.70s)
=== RUN   TestAccAWSWafWebAcl_disappears
--- PASS: TestAccAWSWafWebAcl_disappears (318.02s)
=== RUN   TestAccAWSWafRule_basic
--- PASS: TestAccAWSWafRule_basic (347.32s)
=== RUN   TestAccAWSWafWebAcl_changeNameForceNew
--- PASS: TestAccAWSWafWebAcl_changeNameForceNew (350.59s)

[stevehorsfield]

Wouldn't it be better to defer the mutex unlock?

[radeksimko]

Yes, indeed, thanks.

I had that there before, must have vanished while I was refactoring 🙈

[catsby]

LGTM! 👍

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.