Add support for retention when deleting ApigeeOrganization

[xuchenma]

This PR solves hashicorp/terraform-provider-google#12219

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

apigee: added support for specifying retention when deleting `google_apigee_organization`

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @slevenick, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

---

Your assigned reviewer will help review your code by:

• Ensuring it's backwards compatible, covers common error cases, etc.
• Summarizing the change into a user-facing changelog note.
• Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.

---

[xuchenma]

Hi Sam, this API isn't public yet (will be there later this week) so the tests won't pass for now.
Meanwhile I'm not sure if this is the correct way to add a query parameter that only works for the DELETE call. Could you check please? Thanks!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 21 insertions(+), 1 deletion(-))
Terraform Beta: Diff ( 3 files changed, 170 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2123
Passed tests 1888
Skipped tests: 227
Failed tests: 8

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccFirebaserulesRelease_BasicRelease|TestAccActiveDirectoryDomain_update|TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample|TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample

[slevenick]

It may take some experimentation to find what works for this API. Do you have access to a staging environment?

[modular-magician]

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccFirebaserulesRelease_BasicRelease[view]
TestAccCGCSnippet_eventarcWorkflowsExample[view]

Tests failed during RECORDING mode:
TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample[view]
TestAccActiveDirectoryDomain_update[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[xuchenma]

It may take some experimentation to find what works for this API. Do you have access to a staging environment?

No but with TF_LOG=debug I'm able to verify if the HTTP request is made correctly, so I'm still able to verify if this PR works.

I tried removing "input: true" and "url_param_only: true", but then "retention" is treated as a data field (rather than a query param) in the HTTP request, which returns a 400 bad request:

    "message": "Invalid JSON payload received. Unknown name \"retention\" at 'organization': Cannot find field.",

I'm wondering what is the proper way to treat "retention" as a query param only for DELETE call?

[slevenick]

This is a very unusual field, but yeah query param only seems right.

It definitely cannot be input: true though

[xuchenma]

It seems that with "url_param_only: true" the ApigeeOrganization was created properly, but the delete call was made without the "retention" param:

---[ REQUEST ]---------------------------------------
DELETE /v1/organizations/tf-teste0zf16pcki?alt=json HTTP/1.1
Host: apigee.googleapis.com
User-Agent: Terraform/0.13.6 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google-beta/acc
Content-Type: application/json
Accept-Encoding: gzip

[xuchenma]

The solution here is to add "delete_url" to the resource, set "url_param_only" to true to the field, and skip status check (since LRO will expire when Org is deleted).

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 22 insertions(+), 19 deletions(-))
Terraform Beta: Diff ( 4 files changed, 171 insertions(+), 20 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2130
Passed tests 1889
Skipped tests: 228
Failed tests: 13

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeInstance_networkPerformanceConfig|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccCloudFunctions2Function_fullUpdate|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlUser_mysqlDisabled|TestAccApigeeEnvironmentIamPolicyGenerated|TestAccApigeeEnvironmentIamBindingGenerated|TestAccApigeeEnvironmentIamMemberGenerated

[modular-magician]

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[view]
TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view]
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view]
TestAccCloudFunctions2Function_fullUpdate[view]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccSqlUser_mysqlDisabled[view]
TestAccApigeeEnvironmentIamPolicyGenerated[view]
TestAccApigeeEnvironmentIamBindingGenerated[view]
TestAccApigeeEnvironmentIamMemberGenerated[view]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccComputeInstance_networkPerformanceConfig[view]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 22 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 4 files changed, 171 insertions(+), 12 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2130
Passed tests 1894
Skipped tests: 228
Failed tests: 8

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeInstance_networkPerformanceConfig|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccCloudFunctions2Function_fullUpdate|TestAccCloudRunService_cloudRunServiceStaticOutboundExample

[modular-magician]

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view]
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view]
TestAccCloudFunctions2Function_fullUpdate[view]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccComputeInstance_networkPerformanceConfig[view]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[slevenick]

This looks good at this point. Let me know when the API rolls out and I can test this

[xuchenma]

This looks good at this point. Let me know when the API rolls out and I can test this

This API should be live now. Thanks!

[slevenick]

Tests passed when triggered manually

[xuchenma]

Thank you Sam!