Add a Properties field to ApigeeOrganization

[xuchenma]

This PR fixes hashicorp/terraform-provider-google#10083 by adding a Properties field to ApigeeOrganization.

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

Add a Properties field to ApigeeOrganization.

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@ndmckinley, please review this PR or find an appropriate assignee.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 181 insertions(+), 18 deletions(-))
Terraform Beta: Diff ( 4 files changed, 191 insertions(+), 18 deletions(-))
TF Validator: Diff ( 1 file changed, 62 insertions(+))

[xuchenma]

This PR tries to solve the issue that, running terraform apply to update an existing ApigeeOrganization will remove the "properties" field. (more details: hashicorp/terraform-provider-google#10075)

I have tested with a local build of this PR, the problem still exists, however we now have a workaround.
Details:

# Original ApigeeOrg details, note that there are two properties
xuchenm@xuchenm:~/.terraform/scrip2$ cat main.tf
...
resource "google_apigee_organization" "apigee_org" {
  analytics_region   = "us-central1"
  project_id         = "apigee-tf-test"
  description        = "asdf"
  authorized_network = google_compute_network.apigee_network.id
  depends_on         = [google_service_networking_connection.apigee_vpc_connection]
}
xuchenm@xuchenm:~/.terraform/scrip2$ curl -H "Authorization: Bearer $TOKEN" -H "content-type:application/json" https://apigee.googleapis.com/v1/organizations/apigee-tf-test
{
  "name": "apigee-tf-test",
  "description": "asdf",
  "createdAt": "1639608276778",
  "lastModifiedAt": "1639608277751",
  "properties": {
    "property": [
      {
        "name": "features.mart.connect.enabled",
        "value": "true"
      },
      {
        "name": "features.hybrid.enabled",
        "value": "true"
      }
    ]
  },
  "analyticsRegion": "us-central1",
  "authorizedNetwork": "projects/apigee-tf-test/global/networks/apigee-network",
  "runtimeType": "CLOUD",
  "subscriptionType": "TRIAL",
  "caCertificate": "...",
  "projectId": "apigee-tf-test",
  "state": "ACTIVE",
  "billingType": "EVALUATION",
  "expiresAt": "1644792086159",
  "addonsConfig": {
    "advancedApiOpsConfig": {},
    "integrationConfig": {},
    "monetizationConfig": {}
  }
}

# Update main.tf to modify description of the org.
xuchenm@xuchenm:~/.terraform/scrip2$ vi main.tf

# Apply the change. Note that the "properties" are set to null
xuchenm@xuchenm:~/.terraform/scrip2$ terraform apply
...
Terraform will perform the following actions:

  # google_apigee_organization.apigee_org will be updated in-place
  ~ resource "google_apigee_organization" "apigee_org" {
        analytics_region   = "us-central1"
        authorized_network = "projects/apigee-tf-test/global/networks/apigee-network"
        ca_certificate     = "..."
      ~ description        = "asdf" -> "asdfaa"
        id                 = "organizations/apigee-tf-test"
        name               = "apigee-tf-test"
        project_id         = "apigee-tf-test"
        runtime_type       = "CLOUD"
        subscription_type  = "TRIAL"

      - properties {
          - property {
              - name  = "features.mart.connect.enabled" -> null
              - value = "true" -> null
            }
          - property {
              - name  = "features.hybrid.enabled" -> null
              - value = "true" -> null
            }
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

A workaround is to add the properties explicitly in main.tf:

resource "google_apigee_organization" "apigee_org" {
  analytics_region   = "us-central1"
  project_id         = "apigee-tf-test"
  description        = "asdf"
  properties {
    property {
      name = "features.hybrid.enabled"
      value = "true"
    }
    property {
      name = "features.mart.connect.enabled"
      value = "true"
    }
  }
  authorized_network = google_compute_network.apigee_network.id
  depends_on         = [google_service_networking_connection.apigee_vpc_connection]
}

Then terraform apply won't remove the "properties" anymore:

xuchenm@xuchenm:~/.terraform/scrip2$ terraform apply
...
Terraform will perform the following actions:

  # google_apigee_organization.apigee_org will be updated in-place
  ~ resource "google_apigee_organization" "apigee_org" {
        analytics_region   = "us-central1"
        authorized_network = "projects/apigee-tf-test/global/networks/apigee-network"
        ca_certificate     = "..."
      ~ description        = "asdfaa" -> "asdfabc"
        id                 = "organizations/apigee-tf-test"
        name               = "apigee-tf-test"
        project_id         = "apigee-tf-test"
        runtime_type       = "CLOUD"
        subscription_type  = "TRIAL"

        properties {
            property {
                name  = "features.hybrid.enabled"
                value = "true"
            }
            property {
                name  = "features.mart.connect.enabled"
                value = "true"
            }
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

A question for the reviewer:
Although there's the workaround, is there any way to fix this issue? I noticed ApigeeOrg has many other fields that are not mentioned in main.tf but are also not removed when the org updates; not sure why only "properties" are treated differently.

[nat-henderson]

Although there's the workaround, is there any way to fix this issue?

I would try setting default_from_api: true on the new Properties field, which will use whatever the current value is unless the user specifies it. That's probably the behavior you want.

I noticed ApigeeOrg has many other fields that are not mentioned in main.tf but are also not removed when the org updates; not sure why only "properties" are treated differently.

Yeah - that'll be server-side, internal to google - we aren't doing anything special to zero it out except not sending it! The service team would need to fix that, which seems a lot harder than the approach I suggest!

[xuchenma]

Thanks, I added default_from_api: true to terraform.yaml. Looks like it has worked!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 164 insertions(+))
Terraform Beta: Diff ( 3 files changed, 174 insertions(+))
TF Validator: Diff ( 1 file changed, 62 insertions(+))

[xuchenma]

This PR is on an old branch and has a few conflicts. I've handpicked the relevant changes to another PR: #6464