Make tpgtools generate beta docs into GA as well for parity with mmv1. #5325

nat-henderson · 2021-10-15T00:35:20Z

Also, minor changes to make sure that the docs are a little bit better - allow overrides of documentation values, for instance.

Also, includes the actual description of the resource, which, at the moment is not acceptable in the DCL. Will need to implement an override in the DCL.

Fixes the worst of hashicorp/terraform-provider-google#10252, bringing the urgency down to "we need better docstrings for these fields". I'll get to that soon.

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

all: fixed an issue where some documentation for new resources was not showing up in the GA provider if it was beta-only.

modular-magician · 2021-10-15T00:46:32Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 6 files changed, 101 insertions(+), 8 deletions(-))
Terraform Beta: Diff ( 5 files changed, 14 insertions(+), 14 deletions(-))

rileykarson · 2021-10-15T17:16:42Z

tpgtools/documentation.go

@@ -48,7 +48,7 @@ func mergeResource(res *Resource, resources map[Version][]*Resource, version *Ve
 		}
 		return ga
 	}
-	beta.Description = "Beta only"
+	beta.Description = fmt.Sprintf("Beta only: %s", beta.Description)


Honestly, I'm fairly certain that adding this as-is is worse than having nothing there- "DCL specification for X" is completely meaningless, and there is zero discoverability on what "DCL" means. We should update the description for the affected resources before this part of the change, at least.

Agree - I didn't look at how bad those were. I'll put this change on hold until that's fixed.

rileykarson · 2021-10-15T17:17:25Z

tpgtools/main.go

@@ -90,6 +90,12 @@ func main() {
 		generateResourceFile(resource)
 		generateSweeperFile(resource)
 		generateResourceTestFile(resource)
+	}
+	// Website files are always generated for the beta version.
+	for _, resource := range resources[BETA_VERSION] {


Can we guard this on just the GA version and used the specified version otherwise? w/ the alpha version, we'll want to generate the alpha website.

rileykarson · 2021-10-15T17:22:47Z

tpgtools/overrides/monitoring/samples/monitoredproject/meta.yaml

  - substitution: "project"
    value: ":PROJECT"
+    docs_value: "preexisting-project-to-monitor-with"


You mentioned offline that the DCL provided sample values, but I'm not seeing them in https://github.com/GoogleCloudPlatform/declarative-resource-client-library/blob/main/services/google/monitoring/samples/basic_monitored_project.yaml. Do they not exist? Keeping values for every field in every sample seems onerous.

google-cla · 2021-10-26T21:12:30Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

modular-magician · 2021-10-26T21:25:32Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 140 insertions(+), 45 deletions(-))
Terraform Beta: Diff ( 12 files changed, 51 insertions(+), 49 deletions(-))

modular-magician · 2021-10-26T23:15:38Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceGoogleSQLCaCerts_basic|TestAccAssuredWorkloadsWorkload_BasicHandWritten|TestAccAssuredWorkloadsWorkload_FullHandWritten|TestAccCloudRunService_cloudRunServiceSqlExample|TestAccComputeFirewallPolicyRule_update|TestAccContainerCluster_withAuthenticatorGroupsConfig|TestAccContainerNodePool_withWorkloadIdentityConfig|TestAccDataprocWorkflowTemplate_basic|TestAccEventarcTrigger_BasicHandWritten|TestAccMonitoringMonitoredProject_BasicMonitoredProject|TestAccOrgPolicyPolicy_EnforcePolicy|TestAccOrgPolicyPolicy_ProjectPolicy|TestAccOrgPolicyPolicy_FolderPolicy|TestAccOrgPolicyPolicy_OrganizationPolicy|TestAccPrivatecaCertificateTemplate_BasicCertificateTemplate|TestAccSQLDatabase_sqlDatabaseBasicExample|TestAccSqlDatabaseInstance_basicSecondGen|TestAccSqlDatabaseInstance_basicMSSQL|TestAccSqlDatabaseInstance_settings_basic|TestAccSqlDatabaseInstance_settings_deletionProtection|TestAccSqlDatabaseInstance_slave|TestAccSqlDatabaseInstance_diskspecs|TestAccSqlDatabaseInstance_maintenance|TestAccSqlDatabaseInstance_settings_upgrade|TestAccSqlDatabaseInstance_settingsDowngrade|TestAccSqlDatabaseInstance_authNets|TestAccSqlDatabaseInstance_multipleOperations|TestAccSqlDatabaseInstance_basic_with_user_labels|TestAccSqlDatabaseInstance_withPrivateNetwork|TestAccSqlDatabase_basic|TestAccSqlDatabase_update|TestAccSqlClientCert_mysql|TestAccSqlUser_postgresIAM You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=212770

modular-magician · 2021-10-26T23:58:14Z

Tests failed during RECORDING mode: TestAccOrgPolicyPolicy_EnforcePolicy|TestAccMonitoringMonitoredProject_BasicMonitoredProject|TestAccOrgPolicyPolicy_ProjectPolicy|TestAccEventarcTrigger_BasicHandWritten|TestAccAssuredWorkloadsWorkload_BasicHandWritten|TestAccAssuredWorkloadsWorkload_FullHandWritten|TestAccComputeFirewallPolicyRule_update|TestAccContainerCluster_withAuthenticatorGroupsConfig|TestAccSqlDatabaseInstance_basicMSSQL|TestAccSqlUser_postgresIAM Please fix these to complete your PR

modular-magician · 2021-10-27T00:14:56Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 140 insertions(+), 45 deletions(-))
Terraform Beta: Diff ( 12 files changed, 51 insertions(+), 49 deletions(-))

Also, minor changes to make sure that the docs are a little bit better - allow overrides of documentation values, for instance. Also, includes the actual description of the resource, which, at the moment is not acceptable in the DCL. Will need to implement an override in the DCL.

…ests.

nat-henderson · 2021-10-28T01:32:57Z

Okay - as long as those tests pass overnight, should be good to review. I'll mark it non-draft after checking.

modular-magician · 2021-10-28T01:44:12Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 153 insertions(+), 41 deletions(-))
Terraform Beta: Diff ( 21 files changed, 71 insertions(+), 53 deletions(-))

nat-henderson · 2021-10-28T17:32:55Z

Okay, this looks right to me.

nat-henderson · 2021-10-28T17:36:07Z

Assured workloads and eventarc still have messy docs. Fixing that on the DCL side involves diving into more complicated overrides. If you'll accept this (imo, clear improvement) as is, I'll fix that on our end separately, and we'll get the fix when it can be done on our end.

modular-magician · 2021-11-10T22:30:32Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 155 insertions(+), 41 deletions(-))
Terraform Beta: Diff ( 12 files changed, 69 insertions(+), 48 deletions(-))

modular-magician · 2021-11-11T00:27:24Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDatasourceGoogleServiceNetworkingPeeredDnsDomain_basic|TestAccAssuredWorkloadsWorkload_BasicHandWritten|TestAccAssuredWorkloadsWorkload_FullHandWritten|TestAccCloudFunctionsFunction_vpcConnector|TestAccComputeInstanceFromMachineImage_basic|TestAccComputeInstanceFromMachineImage_overrideMetadataDotStartupScript|TestAccComputeInstanceFromMachineImage_diffProject|TestAccComputeRegionNetworkEndpointGroup_regionNetworkEndpointGroupAppengineExample|TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate|TestAccContainerNodePool_withWorkloadIdentityConfig|TestAccEventarcTrigger_BasicHandWritten|TestAccMonitoringMonitoredProject_BasicMonitoredProject|TestAccOrgPolicyPolicy_EnforcePolicy|TestAccOrgPolicyPolicy_FolderPolicy|TestAccOrgPolicyPolicy_ProjectPolicy|TestAccPrivatecaCertificateTemplate_BasicCertificateTemplate|TestAccSqlDatabaseInstance_withPrivateNetwork|TestAccSqlUser_postgresIAM You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=217340

rileykarson · 2021-11-11T19:06:11Z

There's a failure in TestAccEventarcTrigger_BasicHandWritten that appears related, btw!

nat-henderson · 2021-11-11T19:51:22Z

Nuts, thanks.

GoogleCloudPlatform#5325) * Make tpgtools generate beta docs into GA as well for parity with mmv1. Also, minor changes to make sure that the docs are a little bit better - allow overrides of documentation values, for instance. Also, includes the actual description of the resource, which, at the moment is not acceptable in the DCL. Will need to implement an override in the DCL. * Generate reasonable documentation for autogenerated and handwritten tests. * Default to - instead of _ * Improve documentation further with links and guides.

nat-henderson requested a review from rileykarson October 15, 2021 00:35

google-cla bot added the cla: yes label Oct 15, 2021

rileykarson reviewed Oct 15, 2021

View reviewed changes

nat-henderson marked this pull request as draft October 18, 2021 20:50

nat-henderson mentioned this pull request Oct 20, 2021

Add deserializer for upcoming 'docs_value' field in sample. #5350

Merged

5 tasks

modular-magician mentioned this pull request Oct 20, 2021

Add deserializer for upcoming 'docs_value' field in sample. hashicorp/terraform-provider-google-beta#3748

Merged

5 tasks

google-cla bot added cla: no and removed cla: yes labels Oct 26, 2021

nat-henderson force-pushed the tpgtools-docs branch from 40ea62b to f720a09 Compare October 26, 2021 21:14

google-cla bot added cla: yes and removed cla: no labels Oct 26, 2021

nat-henderson added 4 commits October 27, 2021 18:32

Generate reasonable documentation for autogenerated and handwritten t…

3ca2ad9

…ests.

Default to - instead of _

6f438da

Improve documentation further with links and guides.

6e96c44

nat-henderson force-pushed the tpgtools-docs branch from fd498e8 to 6e96c44 Compare October 28, 2021 01:32

nat-henderson marked this pull request as ready for review October 28, 2021 17:33

rileykarson approved these changes Nov 10, 2021

View reviewed changes

rileykarson mentioned this pull request Nov 10, 2021

Revert "Reimplemented compute service attachment using the DCL (#5342)" #5442

Merged

5 tasks

modular-magician mentioned this pull request Nov 10, 2021

Revert "Reimplemented compute service attachment using the DCL (#5342)" GoogleCloudPlatform/terraform-validator#377

Merged

5 tasks

This was referenced Nov 10, 2021

Revert "Reimplemented compute service attachment using the DCL (#5342)" hashicorp/terraform-provider-google-beta#3847

Merged

Revert "Reimplemented compute service attachment using the DCL (#5342)" hashicorp/terraform-provider-google#10543

Merged

Merge branch 'master' into tpgtools-docs

ba635a3

nat-henderson merged commit a89ee32 into master Nov 10, 2021

nat-henderson deleted the tpgtools-docs branch November 10, 2021 22:46

This was referenced Nov 10, 2021

Make tpgtools generate beta docs into GA as well for parity with mmv1. hashicorp/terraform-provider-google-beta#3848

Merged

Make tpgtools generate beta docs into GA as well for parity with mmv1. hashicorp/terraform-provider-google#10545

Merged

rileykarson mentioned this pull request Nov 12, 2021

Compute serviceattachment dcl #5444

Closed

5 tasks

rileykarson mentioned this pull request Mar 6, 2023

tpgtools should generate beta website files into the GA provider hashicorp/terraform-provider-google#10252

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make tpgtools generate beta docs into GA as well for parity with mmv1. #5325

Make tpgtools generate beta docs into GA as well for parity with mmv1. #5325

nat-henderson commented Oct 15, 2021

modular-magician commented Oct 15, 2021

rileykarson Oct 15, 2021

nat-henderson Oct 18, 2021

rileykarson Oct 15, 2021

rileykarson Oct 15, 2021

google-cla bot commented Oct 26, 2021

modular-magician commented Oct 26, 2021

modular-magician commented Oct 26, 2021

modular-magician commented Oct 26, 2021

modular-magician commented Oct 27, 2021

nat-henderson commented Oct 28, 2021

modular-magician commented Oct 28, 2021

nat-henderson commented Oct 28, 2021

nat-henderson commented Oct 28, 2021

modular-magician commented Nov 10, 2021

modular-magician commented Nov 11, 2021

rileykarson commented Nov 11, 2021

nat-henderson commented Nov 11, 2021

Make tpgtools generate beta docs into GA as well for parity with mmv1. #5325

Make tpgtools generate beta docs into GA as well for parity with mmv1. #5325

Conversation

nat-henderson commented Oct 15, 2021

modular-magician commented Oct 15, 2021

Diff report:

rileykarson Oct 15, 2021

Choose a reason for hiding this comment

nat-henderson Oct 18, 2021

Choose a reason for hiding this comment

rileykarson Oct 15, 2021

Choose a reason for hiding this comment

rileykarson Oct 15, 2021

Choose a reason for hiding this comment

google-cla bot commented Oct 26, 2021

modular-magician commented Oct 26, 2021

Diff report:

modular-magician commented Oct 26, 2021

modular-magician commented Oct 26, 2021

modular-magician commented Oct 27, 2021

Diff report:

nat-henderson commented Oct 28, 2021

modular-magician commented Oct 28, 2021

Diff report:

nat-henderson commented Oct 28, 2021

nat-henderson commented Oct 28, 2021

modular-magician commented Nov 10, 2021

Diff report:

modular-magician commented Nov 11, 2021

rileykarson commented Nov 11, 2021

nat-henderson commented Nov 11, 2021