Upgrade dcl #7012

slevenick · 2022-12-21T01:23:11Z

Upgrade DCL to 1.30.0. Adds support for OSPolicyConfig skip_await_rollout

Disables tests in VCR as they haven't passed successfully in months due to long rollout times. skip_await_rollout won't work in tests due to deletion needing to happen as the last test step, but these resources cannot be deleted until they finish rollout

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

osconfig: added support for `skip_await_rollout` in `google_os_config_os_policy_assignment`

modular-magician · 2022-12-21T01:39:06Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 65 insertions(+), 29 deletions(-))
Terraform Beta: Diff ( 6 files changed, 146 insertions(+), 29 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T02:22:35Z

Tests analytics

Total tests: 2367
Passed tests 2115
Skipped tests: 247
Failed tests: 5

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccFirebaserulesRelease_BasicRelease|TestAccLoggingBucketConfigProject_cmekSettings|TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment|TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment

modular-magician · 2022-12-21T03:04:22Z

Tests passed during RECORDING mode:
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]

Tests failed during RECORDING mode:
TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment[Error message] [Debug log]
TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

slevenick · 2022-12-21T05:11:21Z

/gcbrun cleared out ci

modular-magician · 2022-12-21T05:27:01Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 65 insertions(+), 29 deletions(-))
Terraform Beta: Diff ( 6 files changed, 146 insertions(+), 29 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T06:12:46Z

Tests analytics

Total tests: 2367
Passed tests 2116
Skipped tests: 247
Failed tests: 4

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccLoggingBucketConfigProject_cmekSettings|TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment|TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment

modular-magician · 2022-12-21T06:54:49Z

Tests passed during RECORDING mode:
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]

Tests failed during RECORDING mode:
TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment[Error message] [Debug log]
TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

modular-magician · 2022-12-21T17:52:47Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 201 insertions(+), 1700 deletions(-))
Terraform Beta: Diff ( 6 files changed, 282 insertions(+), 1700 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T18:39:30Z

Tests analytics

Total tests: 2367
Passed tests 2116
Skipped tests: 247
Failed tests: 4

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccLoggingBucketConfigProject_cmekSettings|TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment|TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment|TestAccContainerCluster_withInvalidGatewayApiConfigChannel

modular-magician · 2022-12-21T19:21:19Z

Tests passed during RECORDING mode:
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]

Tests failed during RECORDING mode:
TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment[Error message] [Debug log]
TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

modular-magician · 2022-12-21T19:55:06Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 205 insertions(+), 1704 deletions(-))
Terraform Beta: Diff ( 6 files changed, 286 insertions(+), 1704 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T20:39:20Z

Tests analytics

Total tests: 2370
Passed tests 2118
Skipped tests: 247
Failed tests: 5

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccComputeForwardingRule_update|TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment|TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment|TestAccLoggingBucketConfigProject_cmekSettings

modular-magician · 2022-12-21T21:20:58Z

Tests passed during RECORDING mode:
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccComputeForwardingRule_update[Debug log]
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]

Tests failed during RECORDING mode:
TestAccOsConfigOsPolicyAssignment_PercentOsPolicyAssignment[Error message] [Debug log]
TestAccOsConfigOsPolicyAssignment_FixedOsPolicyAssignment[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

modular-magician · 2022-12-21T22:22:47Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 80 insertions(+), 1936 deletions(-))
Terraform Beta: Diff ( 5 files changed, 80 insertions(+), 1936 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T22:37:52Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 6 files changed, 205 insertions(+), 2850 deletions(-))
Terraform Beta: Diff ( 6 files changed, 205 insertions(+), 2850 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-21T23:24:29Z

Tests analytics

Total tests: 2370
Passed tests 2119
Skipped tests: 249
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccLoggingBucketConfigProject_cmekSettings

modular-magician · 2022-12-21T23:25:46Z

Tests passed during RECORDING mode:
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]

All tests passed
View the build log or the debug log for each test

melinath

In general this LGTM. Will this fix hashicorp/terraform-provider-google#11070? It could be nice to make the test more like a full test by adding more fields, but since it's the same as a test that previously existed I don't think that's a blocker.

The one issue is that this seems to cause the entire documentation page for the resource to get deleted - possibly tpgtools requires at least one sample?

melinath · 2022-12-22T19:00:50Z

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb

+  name     = "tf-test-assignment%{random_suffix}"
+
+  os_policies {
+    id   = "policy"


possibly add allow_no_resource_group_match and description in here to get full coverage?

melinath · 2022-12-22T19:01:25Z

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb

+    id   = "policy"
+    mode = "VALIDATION"
+
+    resource_groups {


possibly add inventory_filters to get full coverage?

melinath · 2022-12-22T19:02:03Z

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb

+
+    resource_groups {
+      resources {
+        id = "apt-to-yum"


possibly add exec, file, and pkg fields & their subfields here?

slevenick · 2022-12-22T19:25:52Z

I'll look into the doc generation. The problem is that these tests fail due to long (1h+) rollout times of the resource because they have to run checks against every instance in the project. They pass locally, but will never pass in CI it seems

The current tests have full coverage but getting that coverage causes many updates (extending test length) and frankly unwieldy configs to show up in the examples. I tried to simplify them and target unused zones to get the rollout to succeed, but even that seems to have failed

rileykarson

re: making the docs unwieldy, doc_hide in the meta.yaml file should help address that?

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb

melinath · 2022-12-22T19:44:52Z

The problem is that these tests fail due to long (1h+) rollout times of the resource because they have to run checks against every instance in the project.

Any chance that we could isolate these tests in their own project(s) to reduce the rollout time?

slevenick · 2022-12-22T20:20:39Z

The problem is that these tests fail due to long (1h+) rollout times of the resource because they have to run checks against every instance in the project.

Any chance that we could isolate these tests in their own project(s) to reduce the rollout time?

Good call! Works locally, testing in CI now

rileykarson

No blockers from me, approving so there aren't too many cooks in the kitchen. Primary review's @melinath.

modular-magician · 2022-12-22T20:40:40Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 6 files changed, 228 insertions(+), 2041 deletions(-))
Terraform Beta: Diff ( 6 files changed, 228 insertions(+), 2041 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-22T21:18:11Z

The provider crashed while running the VCR tests in REPLAYING mode
Please fix it to complete your PR
View the build log

melinath · 2022-12-22T21:40:48Z

It looks like this crash is due to a change merged earlier; I'll make a revert PR.

melinath · 2022-12-22T21:43:39Z

#7032

modular-magician · 2022-12-22T22:07:54Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 6 files changed, 260 insertions(+), 2041 deletions(-))
Terraform Beta: Diff ( 6 files changed, 260 insertions(+), 2041 deletions(-))
TF Validator: Diff ( 2 files changed, 6 insertions(+), 6 deletions(-))

modular-magician · 2022-12-22T22:45:51Z

Tests analytics

Total tests: 2376
Passed tests 2124
Skipped tests: 248
Failed tests: 4

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccOsConfigOsPolicyAssignment_basicOsPolicyAssignment

modular-magician · 2022-12-22T22:50:02Z

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]
TestAccOsConfigOsPolicyAssignment_basicOsPolicyAssignment[Debug log]

All tests passed
View the build log or the debug log for each test

melinath

LGTM

* Upgrade dcl * Add new basic test * Simplify tests * Fix yaml * Use other zone * Update tests * Simplify tests * Remove sample tests, use handwritten * Rename test * Test in separate project, add back docs-only sample * Add test changes too * Add a couple fields

slevenick requested a review from ScottSuarez December 21, 2022 01:23

slevenick added 7 commits December 21, 2022 13:51

Upgrade dcl

78d819d

Add new basic test

777a086

Simplify tests

3950b08

Fix yaml

b3c44f8

Use other zone

a34b4a1

Update tests

752a988

Simplify tests

ec20de4

slevenick force-pushed the ospolicy-skip branch from 16beadb to ec20de4 Compare December 21, 2022 22:05

slevenick added 2 commits December 21, 2022 14:14

Remove sample tests, use handwritten

3848404

Rename test

c0a597b

slevenick requested review from melinath and removed request for ScottSuarez December 22, 2022 17:11

slevenick requested review from rileykarson and removed request for melinath December 22, 2022 19:04

melinath requested changes Dec 22, 2022

View reviewed changes

rileykarson reviewed Dec 22, 2022

View reviewed changes

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb Outdated Show resolved Hide resolved

mmv1/third_party/terraform/tests/resource_os_config_os_policy_assignment_test.go.erb Show resolved Hide resolved

ScottSuarez approved these changes Dec 22, 2022

View reviewed changes

Test in separate project, add back docs-only sample

7fc6927

Add test changes too

4d35b65

rileykarson approved these changes Dec 22, 2022

View reviewed changes

Add a couple fields

11211d9

slevenick force-pushed the ospolicy-skip branch from d7d4146 to 11211d9 Compare December 22, 2022 21:54

melinath approved these changes Dec 22, 2022

View reviewed changes

slevenick merged commit 7f919ab into GoogleCloudPlatform:main Dec 23, 2022

This was referenced Dec 23, 2022

Upgrade dcl hashicorp/terraform-provider-google-beta#5026

Merged

Upgrade dcl hashicorp/terraform-provider-google#13340

Merged

Upgrade dcl GoogleCloudPlatform/terraform-validator#1236

Merged

This was referenced Feb 7, 2023

Failing test(s): Os Policy Assignment: operation not done hashicorp/terraform-provider-google#11070

Closed

Failing test(s): TestAccOsConfigOsPolicyAssignment quota exhausted hashicorp/terraform-provider-google#11502

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade dcl #7012

Upgrade dcl #7012

slevenick commented Dec 21, 2022 •

edited by melinath

Loading

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

slevenick commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

melinath left a comment

melinath Dec 22, 2022

melinath Dec 22, 2022

melinath Dec 22, 2022

slevenick commented Dec 22, 2022

rileykarson left a comment

melinath commented Dec 22, 2022

slevenick commented Dec 22, 2022

rileykarson left a comment

modular-magician commented Dec 22, 2022

modular-magician commented Dec 22, 2022

melinath commented Dec 22, 2022

melinath commented Dec 22, 2022

modular-magician commented Dec 22, 2022

modular-magician commented Dec 22, 2022

modular-magician commented Dec 22, 2022

melinath left a comment

Upgrade dcl #7012

Upgrade dcl #7012

Conversation

slevenick commented Dec 21, 2022 • edited by melinath Loading

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Tests analytics

Action taken

modular-magician commented Dec 21, 2022

slevenick commented Dec 21, 2022

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Tests analytics

Action taken

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Tests analytics

Action taken

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Tests analytics

Action taken

modular-magician commented Dec 21, 2022

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Diff report

modular-magician commented Dec 21, 2022

Tests analytics

Action taken

modular-magician commented Dec 21, 2022

melinath left a comment

Choose a reason for hiding this comment

melinath Dec 22, 2022

Choose a reason for hiding this comment

melinath Dec 22, 2022

Choose a reason for hiding this comment

melinath Dec 22, 2022

Choose a reason for hiding this comment

slevenick commented Dec 22, 2022

rileykarson left a comment

Choose a reason for hiding this comment

melinath commented Dec 22, 2022

slevenick commented Dec 22, 2022

rileykarson left a comment

Choose a reason for hiding this comment

modular-magician commented Dec 22, 2022

Diff report

modular-magician commented Dec 22, 2022

melinath commented Dec 22, 2022

melinath commented Dec 22, 2022

modular-magician commented Dec 22, 2022

Diff report

modular-magician commented Dec 22, 2022

Tests analytics

Action taken

modular-magician commented Dec 22, 2022

melinath left a comment

Choose a reason for hiding this comment

slevenick commented Dec 21, 2022 •

edited by melinath

Loading