Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync main -> 5.0.0 feature branch #9107

Merged
merged 60 commits into from
Sep 27, 2023

Conversation

c2thorn
Copy link
Member

@c2thorn c2thorn commented Sep 27, 2023

Release Note Template for Downstream PRs (will be copied)


burov and others added 30 commits September 19, 2023 14:23
* Added support for project-level custom modules

Related to b/296259216

* gofmt

* Added client-side validation of display_name

* Fixed expressions in update test

* Added mutex
Co-authored-by: Hamza Hassan <hamzahassan@google.com>
* Update membership.go

* Update membership.go
…8930)

* Initial split of diff computation from breaking change computation

* Made a breaking change

* go mod tidy

* Fixed import path

* Corrected function diff detection

* Revert "Made a breaking change"

This reverts commit 0d4fd2b.

* Reverted detection of new required field addition

b/300515447

* Factored out key union logic
…nges (#9008)

* Add upgrade guidance for navigating region/location logic changes

Co-authored-by: BBBmau <alvarez.mauriciotm@gmail.com>

* Move `Resource annotations` section to be nested under `Provider-level Labels Rework` again

---------

Co-authored-by: BBBmau <alvarez.mauriciotm@gmail.com>
* first draft for new Network Attachment resource

* updated NetworkAttachment specification file

* added usage example for Network Attachment with GCE Instance

* verified attributes and descriptions

* enhanced network attachment examples

* adding network attachment field to compute instance network interface

* added network attachment specification to yaml file

* added integration tests

* fixing minor typo

* fixed typo

* fixed typo

* fixed tf config

* fixed typo

* fixed network attachment simple test

* updated instance resource documentation

* removed bad logic network attachment validation

* fixed tests and field specification for network attachment
…9009)

* Add 5.0 guide to delete google_service_networking_connection

* Address comment
* Upload META.d folder

* Update common~copy.yaml
…onfig.sysctls` (#8981)

Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com>
…` blocks (#9048)

* Add guidance for removing empty strings from `provider` blocks

* Update code block to use tf formatting, instead of diff
* adding security policy field to networkInterfaceAccessConfig

* adding security policy to networkInterface instead of networkInterfaceAccessConfig

* finishing solution 1 and adding integration tests and doc

* cleanups for solution 1

* wrapping update security policy for beta

* replacing the networks in tests

* fixing code review by implementing the solution two

* replacing networks in tests

* fixing nic read for instance_template resource

* adding checking for access config security policy while flattening to prevent the instance template to break

* fixing error while creating instance with empty security_policy

* changing region for tests which use network_edge_security_service

* comment all failing tests but one to test if it runs alone

* making the tests running serially

* fixing the tests to be called only by the serial one

* fixing code review comments

* fixing missing compute_image data from merge

* fixing code review
* Add accelerators to workstation update masks

This was missed in #8490, and so in-place updates to accelerators are silently dropped.

Verified the issue by updating my workstation config with accelerators but the actual config did not change. On inspecting the debug logs, I noticed the update mask was wrong. Near the end of the log, the following warning message was present.

```
2023-09-16T04:38:23.768Z [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/google-beta\"]" produced an unexpected new value for google_workstations_workstation_config.sidb, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .etag: was cty.StringVal("...."), but now cty.StringVal("...")
      - .host[0].gce_instance[0].accelerators: block count changed from 1 to 0
```

* Fix test failure
SarahFrench and others added 12 commits September 26, 2023 07:39
* add event_stream block to google_storage_transfer_job schema

* refactor optional/required fields

* add flatteners/expanders

* add flatteners/expanders and tests

* update docs

* remove ForceNew

* add event_stream in update function

* add ConflictsWith for schedule/event_stream

* use ExactlyOneOf and update docs and tests

* Finish event_stream tests

* Update mmv1/third_party/terraform/services/storagetransfer/resource_storage_transfer_job_test.go

Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>

* add randomstrings to config

* remove copyright headers

---------

Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
* fixed import on google_tags_tag_binding

* added d.setid()

---------

Co-authored-by: Edward Sun <sunedward@google.com>
* Removed roaks3 and melinath from vacation list

* Update membership.go
* Transfer tgc/cai2hcl to magic-modules.

This change moves tgc/cai2hcl management to magic-modules/mmv1
using new provider, which will then be used for generated
cai2hcl converters.

* Add cai2hcl generation in generate_downstream.sh

* Remove comment

* Fix cai2hcl bash script

- Add missing -a param
- Move cai2hcl folder name from ruby files to bash

* Fix sh script.

Remove files in cai2hcl/ folder instead of a folder itself.

* Ignore 'tgc-base' step, add TODO

* Fix copyright date

* Remove excessive result map normalization code.

It is not yet needed at this stage for manual converters.

* Rename ConvertOptions back to Options.

Otherwise it breaks current Convert API.

* Change the folder path "google/converters" to
"converters/google/resources", same as in tfplan2cai

* Update cai2hcl folder with latest tgc state.

Once refactoring of terraform-google-conversion is
merged, we we update cai2hcl/ folder so that this PR
moves functionality from tgc to mmv1 verbatim without
any refactorings.
* Added SCC org custom module

* Fixed parallelization of custom module tests

* Converted tabs to spaces
# Conflicts:
#	.ci/containers/membership-checker/membership.go
#	.ci/gcb-community-checker.yml
#	.ci/gcb-contributor-membership-checker.yml
#	.ci/magician/github/membership.go
#	mmv1/products/alloydb/Backup.yaml
#	mmv1/products/secretmanager/Secret.yaml
#	mmv1/products/securitycenter/ProjectCustomModule.yaml
#	mmv1/templates/terraform/examples/scc_project_custom_module_basic.tf.erb
#	mmv1/templates/terraform/examples/scc_project_custom_module_full.tf.erb
#	mmv1/templates/terraform/examples/workstation_config_basic.tf.erb
#	mmv1/third_party/terraform/services/alloydb/resource_alloydb_instance_test.go
#	mmv1/third_party/terraform/services/securitycenter/resource_scc_project_custom_module_test.go
#	mmv1/third_party/terraform/website/docs/guides/version_5_upgrade.html.markdown
@google-cla
Copy link

google-cla bot commented Sep 27, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@modular-magician

This comment was marked as outdated.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3099
Passed tests 2744
Skipped tests: 313
Affected tests: 42

Action taken

Found 42 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccAlloydbBackup_alloydbBackupFullExample|TestAccAlloydbBackup_alloydbBackupBasicExample|TestAccAlloydbInstance_update|TestAccAlloydbBackup_update|TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteAptExample|TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteYumExample|TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_EuropeWest8|TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference|TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_UsEast1|TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_UsCentral1LowerCase|TestAccBigQueryExternalDataTable_connectionIdDiff_UseNameReference|TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy|TestAccComputeSubnetwork_subnetworkPurposePrivateNatExample|TestAccComputeInstance_NetworkAttachmentUpdate|TestAccComputeInstance_NetworkAttachment|TestAccComputeInstance_update|TestAccComputeInstance_PTRRecord|TestAccDataSourceGoogleComputeRouterNat_basic|TestAccComputeRouterNat_withManualIpAndSubnetConfiguration|TestAccComputeRouterNat_withPrivateNatAndRules|TestAccComputeRouterNat_withPortAllocationMethods|TestAccComputeRouterNat_withPrivateNat|TestAccComputeRouterNat_removeLogConfig|TestAccComputeRouterNat_update|TestAccComputeRouterNat_withNatIpsAndDrainNatIps|TestAccComputeRouterNat_withNatRules|TestAccComputeRouterNat_basic|TestAccContainerCluster_withAddons|TestAccContainerNodePool_withLinuxNodeConfig|TestAccOSConfigPatchDeployment_osConfigPatchDeploymentFullExample|TestAccSecretManagerSecret_rotationPeriodUpdate|TestAccSecurityCenterFolderCustomModule|TestAccSecurityCenterProjectCustomModule|TestAccSecurityCenterOrganizationCustomModule|TestAccStorageTransferJob_eventStream|TestAccWorkstationsWorkstationConfig_serviceAccount|TestAccWorkstationsWorkstationConfig_workstationConfigBasicExample|TestAccWorkstationsWorkstationConfigIamMemberGenerated|TestAccWorkstationsWorkstationConfig_update|TestAccWorkstationsWorkstationConfigIamBindingGenerated|TestAccWorkstationsWorkstationConfig_updateHostDetails|TestAccWorkstationsWorkstationConfigIamPolicyGenerated

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccAlloydbInstance_update[Debug log]
TestAccAlloydbBackup_update[Debug log]
TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteAptExample[Debug log]
TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_EuropeWest8[Debug log]
TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference[Debug log]
TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_UsEast1[Debug log]
TestAccBigQueryExternalDataTable_connectionIdDiff_UseIdReference_UsCentral1LowerCase[Debug log]
TestAccBigQueryExternalDataTable_connectionIdDiff_UseNameReference[Debug log]
TestAccComputeSubnetwork_subnetworkPurposePrivateNatExample[Debug log]
TestAccComputeInstance_NetworkAttachmentUpdate[Debug log]
TestAccComputeInstance_NetworkAttachment[Debug log]
TestAccComputeInstance_update[Debug log]
TestAccComputeInstance_PTRRecord[Debug log]
TestAccDataSourceGoogleComputeRouterNat_basic[Debug log]
TestAccComputeRouterNat_withManualIpAndSubnetConfiguration[Debug log]
TestAccComputeRouterNat_withPrivateNatAndRules[Debug log]
TestAccComputeRouterNat_withPortAllocationMethods[Debug log]
TestAccComputeRouterNat_withPrivateNat[Debug log]
TestAccComputeRouterNat_removeLogConfig[Debug log]
TestAccComputeRouterNat_update[Debug log]
TestAccComputeRouterNat_withNatIpsAndDrainNatIps[Debug log]
TestAccComputeRouterNat_withNatRules[Debug log]
TestAccComputeRouterNat_basic[Debug log]
TestAccContainerNodePool_withLinuxNodeConfig[Debug log]
TestAccOSConfigPatchDeployment_osConfigPatchDeploymentFullExample[Debug log]
TestAccSecretManagerSecret_rotationPeriodUpdate[Debug log]
TestAccSecurityCenterProjectCustomModule[Debug log]
TestAccSecurityCenterOrganizationCustomModule[Debug log]
TestAccStorageTransferJob_eventStream[Debug log]
TestAccWorkstationsWorkstationConfig_serviceAccount[Debug log]
TestAccWorkstationsWorkstationConfig_workstationConfigBasicExample[Debug log]
TestAccWorkstationsWorkstationConfigIamMemberGenerated[Debug log]
TestAccWorkstationsWorkstationConfig_update[Debug log]
TestAccWorkstationsWorkstationConfigIamBindingGenerated[Debug log]
TestAccWorkstationsWorkstationConfig_updateHostDetails[Debug log]
TestAccWorkstationsWorkstationConfigIamPolicyGenerated[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbBackup_alloydbBackupFullExample[Error message] [Debug log]
TestAccAlloydbBackup_alloydbBackupBasicExample[Error message] [Debug log]
TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteYumExample[Error message] [Debug log]
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]
TestAccContainerCluster_withAddons[Error message] [Debug log]
TestAccSecurityCenterFolderCustomModule[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@c2thorn
Copy link
Member Author

c2thorn commented Sep 27, 2023

/gcbrun

@modular-magician

This comment was marked as outdated.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Breaking Change(s) Detected

The following breaking change(s) were detected within your pull request.

  • Field effective_labels within resource google_alloydb_backup was either removed or renamed - reference
  • Field metadata.contents_delta_uri changed from optional to required on google_vertex_ai_index - reference
  • Field terraform_labels within resource google_alloydb_backup was either removed or renamed - reference

If you believe this detection to be incorrect please raise the concern with your reviewer. If you intend to make this change you will need to wait for a major release window. An override-breaking-change label can be added to allow merging.

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 62 files changed, 5862 insertions(+), 427 deletions(-))
Terraform Beta: Diff ( 71 files changed, 9064 insertions(+), 1477 deletions(-))
TF Conversion: Diff ( 10 files changed, 993 insertions(+), 17 deletions(-))
TF OiCS: Diff ( 15 files changed, 354 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (14 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  annotations  = # value needed
  display_name = # value needed
}

Resource: google_alloydb_cluster (59 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_cluster" "primary" {
  annotations = # value needed
  etag        = # value needed
}

Resource: google_alloydb_instance (20 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_instance" "primary" {
  query_insights_config {
    query_plans_per_minute  = # value needed
    query_string_length     = # value needed
    record_application_tags = # value needed
    record_client_address   = # value needed
  }
}

Resource: google_compute_instance_from_machine_image (5 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_instance_from_machine_image" "primary" {
  network_interface {
    network_attachment = # value needed
    security_policy    = # value needed
  }
}

Resource: google_compute_instance_from_template (10 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_instance_from_template" "primary" {
  network_interface {
    network_attachment = # value needed
    security_policy    = # value needed
  }
}

Resource: google_compute_router_nat (11 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_router_nat" "primary" {
  rules {
    action {
      source_nat_active_ranges = # value needed
      source_nat_drain_ranges  = # value needed
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3099
Passed tests 2782
Skipped tests: 313
Affected tests: 4

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteYumExample|TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy|TestAccContainerCluster_withAddons|TestAccSecurityCenterFolderCustomModule

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccArtifactRegistryRepository_artifactRegistryRepositoryRemoteYumExample[Error message] [Debug log]
TestAccContainerCluster_withAddons[Error message] [Debug log]
TestAccSecurityCenterFolderCustomModule[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@c2thorn c2thorn requested a review from shuyama1 September 27, 2023 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.