Shared reservation #5547

simamGit · 2021-12-13T23:48:49Z

Added shareSetting to enable SharedReservation creation for compute.reservation.
ShareSetting includes two properties:
-shareType: LOCAL and SPECIFIC_PROJECTS
-project_map: a map of projectId and projectConfig.

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

compute: Added `shareSettings` in `google_compute_reservation`

Merge branch 'sharedReservation' of https://github.com/simamGit/magic-modules into sharedReservation

modular-magician · 2021-12-13T23:50:18Z

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@melinath, please review this PR or find an appropriate assignee.

modular-magician · 2021-12-14T00:01:34Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 297 insertions(+), 40 deletions(-))
Terraform Beta: Diff ( 3 files changed, 257 insertions(+))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 5 files changed, 123 insertions(+))

melinath · 2021-12-15T21:32:45Z

/gcbrun

modular-magician · 2021-12-15T21:44:56Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 257 insertions(+))
Terraform Beta: Diff ( 4 files changed, 275 insertions(+), 18 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 5 files changed, 123 insertions(+))

modular-magician · 2021-12-15T23:30:02Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDatasourceGoogleServiceNetworkingPeeredDnsDomain_basic|TestAccApigeeEnvironmentIamMemberGenerated|TestAccCloudFunctionsFunction_vpcConnector|TestAccComputeInstance_reservationAffinities|TestAccComputeReservation_sharedReservationBasicExample|TestAccComputeReservation_reservationBasicExample|TestAccComputeReservation_update|TestAccContainerNodePool_withInvalidUpgradeSettings You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=232210

paul-hoang · 2021-12-16T00:51:25Z

@melinath Could you also add me as a reviewer for this pull request? (I'm new to this process, so please feel free to let me know if this is not needed).

melinath · 2021-12-16T16:17:43Z

@paul-hoang I can't add you as a reviewer, but I think you may be able to provide a review regardless.

melinath

It looks like TestAccComputeReservation_sharedReservationBasicExample is failing with the following error:

Error: Error creating Reservation: googleapi: Error 412: Constraint constraints/compute.sharedReservationsOwnerProjects violated for project projects/<project-number>
Shared reservation can not be created in this project for your organization., conditionNotMet

Please ensure that this test passes locally & then re-request review.

simamGit · 2021-12-16T20:52:51Z

It looks like TestAccComputeReservation_sharedReservationBasicExample is failing with the following error:
Error: Error creating Reservation: googleapi: Error 412: Constraint constraints/compute.sharedReservationsOwnerProjects violated for project projects/<project-number>
Shared reservation can not be created in this project for your organization., conditionNotMet
Please ensure that this test passes locally & then re-request review.

ran reservation tests successfully before PR. I can locally run both reservation and sharedreservation tests successfully. using these two commands:
TF_LOG=DEBUG make testacc TEST=./google TESTARGS=' run=TestAccComputeReservation_sharedReservationBasicExample'

TF_LOG=DEBUG make testacc TEST=./google TESTARGS='-run=TestAccComputeReservation_reservationBasicExample'

but it is also set the environment variables for these tests. specially project play key role for these tests. I set these three EVN variables: GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_PROJECT , GOOGLE_REGION

so the question here is which environment these tests are running and how can i set them on test time?

modular-magician · 2021-12-20T19:46:42Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 291 insertions(+), 31 deletions(-))
Terraform Beta: Diff ( 5 files changed, 278 insertions(+), 18 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 5 files changed, 123 insertions(+))

melinath · 2021-12-22T21:25:53Z

/gcbrun

modular-magician · 2021-12-22T21:38:49Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 260 insertions(+))
Terraform Beta: Diff ( 5 files changed, 278 insertions(+), 18 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 5 files changed, 123 insertions(+))

simamGit · 2021-12-23T01:14:54Z

It looks like TestAccComputeReservation_sharedReservationBasicExample is failing with the following error:
Error: Error creating Reservation: googleapi: Error 412: Constraint constraints/compute.sharedReservationsOwnerProjects violated for project projects/<project-number>
Shared reservation can not be created in this project for your organization., conditionNotMet
Please ensure that this test passes locally & then re-request review.

Hi Stephen,
I updated the sharedReservation test to cover all required environment setting/configuration for test. This test starts with creating two projects and set the org policy and finally creates sharedReservation.

to create project and set org policy the user needs to have these roles: Organization Administrator, Organization Policy Administrator and Project Creator.

also we need to set these environment variables
GOOGLE_PROJECT, GOOGLE_REGION, GOOGLE_ZONE,GOOGLE_BILLING_ACCOUNT.
in my case since I used the gcloud authentication I also set GOOGLE_USE_DEFAULT_CREDENTIALS=true.

It is passing successfully in my local machine but all these new changes are for Terraform environment. hopefully this time it can pass in Terrafom Environment.

modular-magician · 2021-12-23T01:19:46Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 285 insertions(+), 18 deletions(-))
Terraform Beta: Diff ( 5 files changed, 285 insertions(+), 18 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 1 file changed, 3 insertions(+))

melinath · 2021-12-23T16:11:49Z

/gcbrun

modular-magician · 2021-12-23T16:24:13Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 267 insertions(+))
Terraform Beta: Diff ( 4 files changed, 267 insertions(+))
TF Validator: Diff ( 1 file changed, 67 insertions(+))
TF OiCS: Diff ( 1 file changed, 3 insertions(+))

modular-magician · 2021-12-23T18:10:41Z

Tests were added that did not run in TeamCity:

TestAccComputeReservation_sharedReservationBasicExample

modular-magician · 2021-12-23T18:10:41Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccApigeeEnvironmentIamBindingGenerated|TestAccApigeeEnvironmentIamPolicyGenerated|TestAccCloudFunctionsFunction_vpcConnector|TestAccComposerEnvironment_composerV1MasterAuthNetworksUpdate|TestAccComposerEnvironment_composerV2MasterAuthNetworksUpdate|TestAccComputeInstance_reservationAffinities|TestAccComputeReservation_update|TestAccContainerNodePool_withInvalidUpgradeSettings You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=235482

melinath · 2021-12-23T19:08:23Z

I'm running the new test manually in Beta since it skips vcr: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/235485

simamGit · 2021-12-23T19:16:33Z

I'm running the new test manually in Beta since it skips vcr: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/235485

Thanks, it seems the main tests sharedReservation and reservation are passing ( i do not see them in failing list). most of the tests which are failing does not look related. I am going to check the TestAccComputeInstance_reservationAffinities, TestAccComputeReservation_update to see if their failing is due to my recent changes.

melinath · 2021-12-23T19:27:59Z

I am going to check the TestAccComputeInstance_reservationAffinities, TestAccComputeReservation_update to see if their failing is due to my recent changes.

Thanks for pointing these out as potential issues; I've started a new run in Teamcity for just these tests as well: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/235487

simamGit · 2021-12-23T21:42:38Z

I am going to check the TestAccComputeInstance_reservationAffinities, TestAccComputeReservation_update to see if their failing is due to my recent changes.

Thanks for pointing these out as potential issues; I've started a new run in Teamcity for just these tests as well: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/235487

I ran locally the reservation related tests:
TF_LOG=DEBUG make testacc TEST=./google TESTARGS='-run=TestAccComputeReservation_'
just in case:
TF_LOG=DEBUG make testacc TEST=./google TESTARGS='-run=TestAccComputeReservation_update'
all are passing.

melinath

@simamGit +1 it looks like TestAccComputeReservation_ is passing in CI as well. However, TestAccComputeInstance_reservationAffinities seems to be failing with a permadiff:

Terraform will perform the following actions:
# google_compute_reservation.reservation must be replaced
-/+ resource "google_compute_reservation" "reservation" {
+ commitment                    = (known after apply)
~ creation_timestamp            = "2021-12-23T11:32:06.647-08:00" -> (known after apply)
~ id                            = "projects/project-id/zones/us-central1-a/reservations/tf-test-resaffinity-et2yep0tb1" -> (known after apply)
name                          = "tf-test-resaffinity-et2yep0tb1"
~ project                       = "project-id" -> (known after apply)
~ self_link                     = "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-central1-a/reservations/tf-test-resaffinity-et2yep0tb1" -> (known after apply)
specific_reservation_required = true
~ status                        = "READY" -> (known after apply)
~ zone                          = "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-central1-a" -> "us-central1-a"
- share_settings {
- share_type = "LOCAL" -> null
}
~ specific_reservation {
count        = 1
~ in_use_count = 0 -> (known after apply)
~ instance_properties {
machine_type     = "n1-standard-1"
+ min_cpu_platform = (known after apply)
}
}
}

The relevant part is:

- share_settings {
- share_type = "LOCAL" -> null
}

This means that the API returned a value for this field that Terraform didn't expect, so Terraform is trying to set it to null. I believe you can work around this by adding default_from_api to shareSettings and shareType in terraform.yaml - that will make Terraform accept whatever the API returns.

modular-magician · 2022-01-05T20:31:41Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 262 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 9 files changed, 285 insertions(+), 27 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))

melinath · 2022-01-05T21:16:34Z

It looks like there's an issue on the API side; please re-request review once that's been resolved. I think everything here should be ready to go once that's finished. https://google.aip.dev/cloud/2510

modular-magician · 2022-01-06T19:25:15Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 281 insertions(+), 22 deletions(-))
Terraform Beta: Diff ( 8 files changed, 264 insertions(+), 5 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))

modular-magician · 2022-01-07T00:58:22Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 221 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 8 files changed, 222 insertions(+), 6 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))

melinath · 2022-01-11T23:20:31Z

Beta test run for just TestAccComputeReservation_sharedReservationBasicExample|TestAccComputeInstance_reservationAffinities|TestAccComputeReservation_update: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaGoogleProject/242318

melinath · 2022-01-12T18:21:16Z

/gcbrun

modular-magician · 2022-01-12T18:33:57Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 243 insertions(+), 27 deletions(-))
Terraform Beta: Diff ( 8 files changed, 222 insertions(+), 6 deletions(-))
TF Validator: Diff ( 1 file changed, 67 insertions(+))

melinath · 2022-01-12T19:08:37Z

Per discussion, we're moving forward with ignore_read rather than fixing the API issue. ignore_read doesn't seem to work as expected inside the map object, so it has to be set to ignore reads on all of share settings.

melinath · 2022-01-12T19:09:43Z

Trying a beta run again - the last one failed for unrelated reasons: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaGoogleProject/242816

melinath

This LGTM as long as the tests pass.

paul-hoang · 2022-01-12T19:31:20Z

LGTM.

modular-magician · 2022-01-12T20:24:16Z

Tests were added that did not run in TeamCity:

TestAccComputeReservation_sharedReservationBasicExample

modular-magician · 2022-01-12T20:24:17Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDatasourceGoogleServiceNetworkingPeeredDnsDomain_basic|TestAccApigeeEnvironmentIamBindingGenerated|TestAccApigeeEnvironmentIamPolicyGenerated|TestAccComputeReservation_reservationBasicExample|TestAccComputeReservation_update|TestAccContainerCluster_withDNSConfig|TestAccContainerNodePool_withInvalidUpgradeSettings|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccPrivatecaCertificate_privatecaCertificateConfigExample|TestAccPrivatecaCertificate_privatecaCertificateNoAuthorityExample You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=242828

melinath · 2022-01-12T21:28:33Z

new beta test run for TestAccComputeReservation_sharedReservationBasicExample|TestAccComputeInstance_reservationAffinities|TestAccComputeReservation_update: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/242833

I was running the other one in the wrong place which is why it was failing 🤦

* Adding shareSetting to create SherdReservations. * Add ShareSetting to create sharedReservation. * fixing the update test * fixing sharedReservation test * adding shareType local to instance test. * fixing shareType issue. * fixing shareType issue. * fix ShareType issue. * fix shareType issue. * unde recent changes. * provide digit validation for project_number. * set ignore_read=true for shareSettings to enable project_id.

simamGit added 3 commits December 13, 2021 21:54

Adding shareSetting to create SherdReservations.

7eb4be2

Add ShareSetting to create sharedReservation.

b41cd39

Adding shareSetting to enable SharedReservation.

3580fb8

Merge branch 'sharedReservation' of https://github.com/simamGit/magic-modules into sharedReservation

modular-magician requested a review from melinath December 13, 2021 23:50

melinath requested changes Dec 16, 2021

View reviewed changes

melinath self-requested a review December 16, 2021 21:22

fixing the update test

6067085

fixing sharedReservation test

846a9bb

melinath requested changes Dec 23, 2021

View reviewed changes

simamGit added 4 commits January 5, 2022 19:49

fixing shareType issue.

cb9655b

fix ShareType issue.

0cc878b

fix shareType issue.

7ec2855

unde recent changes.

2dba174

provide digit validation for project_number.

6f79004

set ignore_read=true for shareSettings to enable project_id.

03ca542

melinath self-requested a review January 7, 2022 16:40

melinath approved these changes Jan 12, 2022

View reviewed changes

melinath merged commit 2bbfb88 into GoogleCloudPlatform:master Jan 12, 2022

This was referenced Jan 12, 2022

Shared reservation GoogleCloudPlatform/terraform-validator#464

Merged

Shared reservation hashicorp/terraform-provider-google#10899

Merged

Shared reservation hashicorp/terraform-provider-google-beta#3980

Merged

rileykarson mentioned this pull request Apr 25, 2022

Support Shared Reservations with Terraform (beta) hashicorp/terraform-provider-google#9790

Closed

Shared reservation #5547

Shared reservation #5547

Conversation

simamGit commented Dec 13, 2021 • edited Loading

modular-magician commented Dec 13, 2021

modular-magician commented Dec 14, 2021

Diff report:

melinath commented Dec 15, 2021

modular-magician commented Dec 15, 2021

Diff report:

modular-magician commented Dec 15, 2021

paul-hoang commented Dec 16, 2021

melinath commented Dec 16, 2021

melinath left a comment

Choose a reason for hiding this comment

simamGit commented Dec 16, 2021 • edited Loading

modular-magician commented Dec 20, 2021

Diff report:

melinath commented Dec 22, 2021

modular-magician commented Dec 22, 2021

Diff report:

simamGit commented Dec 23, 2021 • edited Loading

modular-magician commented Dec 23, 2021

Diff report:

melinath commented Dec 23, 2021

modular-magician commented Dec 23, 2021

Diff report:

modular-magician commented Dec 23, 2021

modular-magician commented Dec 23, 2021

melinath commented Dec 23, 2021

simamGit commented Dec 23, 2021

melinath commented Dec 23, 2021

simamGit commented Dec 23, 2021

melinath left a comment

Choose a reason for hiding this comment

modular-magician commented Jan 5, 2022

Diff report:

melinath commented Jan 5, 2022

modular-magician commented Jan 6, 2022

Diff report:

modular-magician commented Jan 7, 2022

Diff report:

melinath commented Jan 11, 2022

melinath commented Jan 12, 2022

modular-magician commented Jan 12, 2022

Diff report:

melinath commented Jan 12, 2022

melinath commented Jan 12, 2022

melinath left a comment

Choose a reason for hiding this comment

paul-hoang commented Jan 12, 2022

modular-magician commented Jan 12, 2022

modular-magician commented Jan 12, 2022

melinath commented Jan 12, 2022

simamGit commented Dec 13, 2021 •

edited

Loading

simamGit commented Dec 16, 2021 •

edited

Loading

simamGit commented Dec 23, 2021 •

edited

Loading