Failing test(s): TestAccApigeeOrganization_apigeeOrganizationCloud* (some Beta-only)

[SarahFrench]

Failure rates

• 22-65% since 2023-08-12

Impacted tests

• TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample
• TestAccApigeeOrganization_apigeeOrganizationCloudFullTestExample (Beta)
• TestAccApigeeOrganization_apigeeOrganizationCloudFullDisableVpcPeeringTestExample

All flagged as flakey tests

Affecte d Resource(s)

• google_apigee_organization

Nightly build test history

• Link

Message(s)

Two errors that are both different presentations of ordering issues in the properties field

Copy to clipboard
------- Stdout: -------
=== RUN   TestAccApigeeOrganization_apigeeOrganizationCloudFullDisableVpcPeeringTestExample
=== PAUSE TestAccApigeeOrganization_apigeeOrganizationCloudFullDisableVpcPeeringTestExample
=== CONT  TestAccApigeeOrganization_apigeeOrganizationCloudFullDisableVpcPeeringTestExample
    vcr_utils.go:152: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:
        Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place
        Terraform will perform the following actions:
          # google_apigee_organization.org will be updated in-place
          ~ resource "google_apigee_organization" "org" {
                id                                   = "organizations/tf-testa3krtayasv"
                name                                 = "tf-testa3krtayasv"
                # (12 unchanged attributes hidden)
              ~ properties {
                  ~ property {
                      ~ name  = "features.hybrid.enabled" -> "features.mart.connect.enabled"
                        # (1 unchanged attribute hidden)
                    }
                  ~ property {
                      ~ name  = "features.mart.connect.enabled" -> "features.hybrid.enabled"
                        # (1 unchanged attribute hidden)
                    }
                }
            }
        Plan: 0 to add, 1 to change, 0 to destroy.
--- FAIL: TestAccApigeeOrganization_apigeeOrganizationCloudFullDisableVpcPeeringTestExample (451.22s)
FAIL

------- Stdout: -------
=== RUN   TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample
=== PAUSE TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample
=== CONT  TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample
    vcr_utils.go:152: ImportStateVerify attributes not equivalent. Difference is shown below. Top is actual, bottom is expected.
        (map[string]string) (len=2) {
         (string) (len=28) "properties.0.property.0.name": (string) (len=23) "features.hybrid.enabled",
         (string) (len=28) "properties.0.property.1.name": (string) (len=29) "features.mart.connect.enabled"
        }
        (map[string]string) (len=2) {
         (string) (len=28) "properties.0.property.0.name": (string) (len=29) "features.mart.connect.enabled",
         (string) (len=28) "properties.0.property.1.name": (string) (len=23) "features.hybrid.enabled"
        }
--- FAIL: TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample (749.36s)
FAIL

[SarahFrench]

I think this can be partially addressed by doing a state-verify-ignore on this field - that'll address the ImportStateVerify error shown above.

But that won't address the issue of non-empty plans due to the API ordering things differently.

[SarahFrench]

👆 created a bug report for this, as the solution might be to change the field to a set in a future breaking change.

I'll update the test to ignore that field when verifying the import step, but feels like this test will keep failing due to the underlying issue causing non-empty plans too. To do this I'd need to convert a bunch of generated tests to handwritten ones - I'll ask for guidance. Edit: No I don't! I can use ignore_read_extra. Yay!

[SarahFrench]

These tests have passed since merging the above PR

• TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample (GA/Beta)
• TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample (GA/Beta)

I realised that I missed the Beta-only test, so I've opened this PR here: GoogleCloudPlatform/magic-modules#9624

[SarahFrench]

Last PR is merged - I'll check on this issue after the holidays and close it

[melinath]

Duplicate of #13274

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.