chore: generate types & mappers for RedisCluster #2289

justinsb · 2024-07-13T14:06:09Z

This is (mostly) generated, and then we pull out a few types we want to override.

There is some duplicate generation of functions and types, detecting that and skipping duplicates that is in a separate branch/PR.

justinsb · 2024-07-17T23:42:54Z

pkg/controller/direct/redis/cluster/mapper.go

+	"google.golang.org/protobuf/types/known/timestamppb"
+)
+
+func Cluster_CreateTime_FromProto(mapCtx *MapContext, in *timestamppb.Timestamp) *string {


I think we may be able to automatically generate these (or automatically generate them, then allow overrides) ... but I want to collect a few more examples first

justinsb · 2024-07-17T23:54:46Z

dev/tools/proto-to-mapper/main.go

@@ -755,8 +755,10 @@ func (v *visitor) writeMapFunctions() {
 			v.generatedFiles[k] = out

 			out.contents.WriteString(fmt.Sprintf("package %s\n\n", lastGoComponent(goPackage)))
-			out.contents.WriteString(fmt.Sprintf("import pb %q\n\n", "cloud.google.com/go/monitoring/dashboard/apiv1/dashboardpb"))
-			out.contents.WriteString(fmt.Sprintf("import krm %q\n\n", "github.com/GoogleCloudPlatform/k8s-config-connector/apis/monitoring/v1beta1"))
+			out.contents.WriteString("import (\n")


This is what gofmt does, saves us a gofmt (as do some of the tab changes)

justinsb · 2024-07-17T23:55:10Z

dev/tools/controllerbuilder/template/apis/doc.go

-{{- if .PackageProtoTag }}
-// +kcc:proto={{ .PackageProtoTag }}
-{{- end }}
+{{ if .PackageProtoTag }}// +kcc:proto={{ .PackageProtoTag }}{{ end }}


Avoiding an awkward (lack of) line break

justinsb · 2024-07-17T23:55:34Z

dev/tools/controllerbuilder/pkg/commands/generatetypes/generatetypescommand.go

 			return "", false
 		}
-
-		protoPackagePath = string(msg.ParentFile().Package())


We're now driven much more by the flags

justinsb · 2024-07-17T23:55:47Z

dev/tools/controllerbuilder/pkg/commands/generatemapper/generatemappercommand.go

@@ -93,7 +93,7 @@ func RunGenerateMapper(ctx context.Context, o *GenerateMapperOptions) error {
 		if strings.HasSuffix(fullName, "Response") {
 			return "", false
 		}
-		if !strings.HasPrefix(fullName, o.ServiceName) {
+		if !strings.HasPrefix(fullName, o.ServiceName+".") {


Avoids foo.v1alpha1 matching foo.v1 !

justinsb · 2024-07-17T23:57:01Z

dev/tools/controllerbuilder/pkg/codegen/mappergenerator.go

@@ -499,7 +509,12 @@ func (v *MapperGenerator) writeMapFunctionsForPair(out io.Writer, pair *typePair
 				}

 				oneof := protoField.ContainingOneof()
-				if oneof != nil {
+				if protoField.HasOptionalKeyword() {


A lot of this complexity is because we are guessing the go field types. It might easier to actually look at the target go type, and it might also save us some overrides!

justinsb · 2024-07-17T23:57:28Z

dev/tools/controllerbuilder/pkg/codegen/mappergenerator.go

+			out.contents.WriteString("import (\n")
+			out.contents.WriteString(fmt.Sprintf("\tpb %q\n", pbPackage))
+			out.contents.WriteString(fmt.Sprintf("\tkrm %q\n", krmPackage))
+			out.contents.WriteString(fmt.Sprintf("\t%q\n", "github.com/GoogleCloudPlatform/k8s-config-connector/pkg/controller/direct"))


All this direct stuff is using the code that @acpana just refactored out 🎉

justinsb · 2024-07-17T23:58:13Z

pkg/test/fuzz/generate.go

+		if count > 4 {
+			return
+		}
+		// Generate a "reasonable" timestamp


"Unreasonable" dates are (literally) billions of years in the future. Not a problem in practice, but the fuzzer finds these problems pretty fast if we don't stop them!

justinsb · 2024-07-17T23:59:24Z

pkg/controller/direct/redis/cluster/roundtrip_test.go

+		)
+
+		// Status fields
+		unimplementedFields.Insert(".discovery_endpoints")


This file is not currently generated. I think it would be nice to collect more examples and then automate. e.g. I think we can tell which fields are spec and which are status by seeing which ones map to spec and which to status (and if they map to neither, that's an error)

justinsb · 2024-07-18T00:00:26Z

pkg/controller/direct/redis/cluster/mapper.go

+	"google.golang.org/protobuf/types/known/timestamppb"
+)
+
+func Cluster_CreateTime_FromProto(mapCtx *direct.MapContext, in *timestamppb.Timestamp) *string {


We should be able to automatically generate these in future (particularly once we're able to skip generation if there is a manually-written function with the same name). But I want to collect more examples first.

cheftako · 2024-07-18T18:19:56Z

dev/tools/controllerbuilder/pkg/codegen/mappergenerator.go

+			out.contents.WriteString("import (\n")
+			out.contents.WriteString(fmt.Sprintf("\tpb %q\n", pbPackage))
+			out.contents.WriteString(fmt.Sprintf("\tkrm %q\n", krmPackage))
+			out.contents.WriteString(fmt.Sprintf("\t%q\n", "github.com/GoogleCloudPlatform/k8s-config-connector/pkg/controller/direct"))


Why is this a string we need to manually validate? Wouldn't it be better to use direct.GoPackage ?

Not sure I follow - did this comment get moved between lines?

cheftako · 2024-07-18T18:32:39Z

dev/tools/controllerbuilder/template/apis/doc.go

@@ -1,7 +1,6 @@
 package apis

-const DocTemplate = `
-// Copyright 2024 Google LLC
+const DocTemplate = `// Copyright 2024 Google LLC


Would it make sense to construct DocTemplate and GroupVersionInfoTemplate from a new CopyRightTemplate? This both allows us to only fix the copyright in 1 place. It also emphasizes what is different and what is the same between them.

Yeah, we can also automatically add the copyright to the file as part of writing it ... I think that's in a PR that's coming!

cheftako · 2024-07-18T18:36:55Z

/lgtm
/approve

Flagged a couple of spots I think we may want to improve but nothing which needs to hold this up.

google-oss-prow · 2024-07-18T18:37:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cheftako

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cheftako]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

justinsb force-pushed the generate_for_rediscluster branch 2 times, most recently from 62b3d97 to fa9c6fa Compare July 17, 2024 23:39

justinsb changed the title ~~generate for rediscluster~~ chore: generate types & mappers for RedisCluster Jul 17, 2024

justinsb commented Jul 17, 2024

View reviewed changes

justinsb force-pushed the generate_for_rediscluster branch from fa9c6fa to 0125ec2 Compare July 17, 2024 23:52

justinsb commented Jul 17, 2024

View reviewed changes

justinsb commented Jul 18, 2024

View reviewed changes

justinsb force-pushed the generate_for_rediscluster branch 2 times, most recently from 4d095ef to acd5abd Compare July 18, 2024 11:35

chore: generate types & mappers for RedisCluster

47ef09e

justinsb force-pushed the generate_for_rediscluster branch from acd5abd to 47ef09e Compare July 18, 2024 11:44

cheftako reviewed Jul 18, 2024

View reviewed changes

google-oss-prow bot assigned cheftako Jul 18, 2024

google-oss-prow bot added the lgtm label Jul 18, 2024

google-oss-prow bot added the approved label Jul 18, 2024

google-oss-prow bot merged commit 6df3bad into GoogleCloudPlatform:master Jul 18, 2024
13 of 14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: generate types & mappers for RedisCluster #2289

chore: generate types & mappers for RedisCluster #2289

justinsb commented Jul 13, 2024 •

edited

Loading

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 17, 2024

justinsb Jul 18, 2024

cheftako Jul 18, 2024

justinsb Jul 19, 2024

cheftako Jul 18, 2024

justinsb Jul 19, 2024

cheftako commented Jul 18, 2024

google-oss-prow bot commented Jul 18, 2024

chore: generate types & mappers for RedisCluster #2289

chore: generate types & mappers for RedisCluster #2289

Conversation

justinsb commented Jul 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cheftako commented Jul 18, 2024

google-oss-prow bot commented Jul 18, 2024

justinsb commented Jul 13, 2024 •

edited

Loading