Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to handle scream snake case enums. #1529

Merged
merged 3 commits into from
Jun 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 53 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,17 @@ See [contrib/tools/workflowcheck](contrib/tools/workflowcheck) for a tool to det
## Contributing
We'd love your help in making the Temporal Go SDK great. Please review our [contribution guidelines](CONTRIBUTING.md).

## Go build and run tags
## Go SDK upgrading past v1.25.1

Go SDK version v1.26.0 switched from using https://github.com/gogo/protobuf to https://github.com/golang/protobuf. While this migration is mostly internal there are a few user visible changes to be aware of:

### Change in types

* `time.Time` in proto structs will now be [timestamppb.Timestamp](https://pkg.go.dev/google.golang.org/protobuf@v1.31.0/types/known/timestamppb#section-documentation)
* `time.Duration` will now be [durationpb.Duration](https://pkg.go.dev/google.golang.org/protobuf/types/known/durationpb)
* V2-generated structs embed locks, so you cannot dereference them.

### Invalid UTF-8

Prior to SDK version v1.26.0 our protobuf code generator allowed invalid UTF-8 data to be stored as proto strings. This isn't actually allowed by the proto3 spec, so if you're using our SDK and think you may store arbitrary binary data in our strings you should set `-tags protolegacy` when building against our SDK.

Expand All @@ -67,5 +77,47 @@ If you see an error like `grpc: error unmarshalling request: string field contai

If you're unsure then you should specify it anyways as there's no harm in doing so unless you relied on the protobuf compiler to ensure all strings were valid UTF-8.

### Incompatible proto/json encoding

Proto enums will, when formatted to JSON, now be in SCREAMING_SNAKE_CASE rather than PascalCase.
* If trying to deserialize old JSON with PascalCase to proto use [go.temporal.io/api/temporalproto]

If users used Temporal proto types in their Workflows, such as for activity output, users may need to modify the default data converter to handle these payloads.
``` go
converter.NewProtoJSONPayloadConverterWithOptions(converter.ProtoJSONPayloadConverterOptions{
LegacyTemporalProtoCompat: true,
}),
```

While upgrading from Go SDK version `< 1.26.0` to a version `>= 1.26.0` users may want to also bias towards using
proto binary to avoid any potential incompatibilities due to having clients serialize messages with incompatible `proto/json` format.

On clients running Go SDK `< 1.26.0`
``` go
converter.NewCompositeDataConverter(
converter.NewNilPayloadConverter(),
converter.NewByteSlicePayloadConverter(),
converter.NewProtoPayloadConverter(),
converter.NewProtoJSONPayloadConverterWithOptions(),
converter.NewJSONPayloadConverter(),
)
```

On clients running Go SDK `>= 1.26.0`

``` go
converter.NewCompositeDataConverter(
converter.NewNilPayloadConverter(),
converter.NewByteSlicePayloadConverter(),
converter.NewProtoPayloadConverter(),
converter.NewProtoJSONPayloadConverterWithOptions(converter.ProtoJSONPayloadConverterOptions{
LegacyTemporalProtoCompat: true,
}),
converter.NewJSONPayloadConverter(),
)
```

Note: Payloads encoded with `proto/binary` will not be readable in the Temporal web UI.

## License
MIT License, please see [LICENSE](LICENSE) for details.
34 changes: 24 additions & 10 deletions converter/proto_json_payload_converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,19 @@ import (
gogojsonpb "github.com/gogo/protobuf/jsonpb"
gogoproto "github.com/gogo/protobuf/proto"
commonpb "go.temporal.io/api/common/v1"
"go.temporal.io/api/temporalproto"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
)

// ProtoJSONPayloadConverter converts proto objects to/from JSON.
type ProtoJSONPayloadConverter struct {
gogoMarshaler gogojsonpb.Marshaler
gogoUnmarshaler gogojsonpb.Unmarshaler
protoMarshalOptions protojson.MarshalOptions
protoUnmarshalOptions protojson.UnmarshalOptions
options ProtoJSONPayloadConverterOptions
gogoMarshaler gogojsonpb.Marshaler
gogoUnmarshaler gogojsonpb.Unmarshaler
protoMarshalOptions protojson.MarshalOptions
protoUnmarshalOptions protojson.UnmarshalOptions
temporalProtoUnmarshalOptions temporalproto.CustomJSONUnmarshalOptions
options ProtoJSONPayloadConverterOptions
}

// ProtoJSONPayloadConverterOptions represents options for `NewProtoJSONPayloadConverterWithOptions`.
Expand All @@ -64,6 +66,10 @@ type ProtoJSONPayloadConverterOptions struct {

// EmitUnpopulated specifies whether to emit unpopulated fields.
EmitUnpopulated bool

// LegacyTemporalProtoCompat will allow enums serialized as SCREAMING_SNAKE_CASE.
// Useful for backwards compatibility when migrating a proto message from gogoproto to standard protobuf.
LegacyTemporalProtoCompat bool
}

var (
Expand All @@ -73,10 +79,11 @@ var (
// NewProtoJSONPayloadConverter creates new instance of `ProtoJSONPayloadConverter`.
func NewProtoJSONPayloadConverter() *ProtoJSONPayloadConverter {
return &ProtoJSONPayloadConverter{
gogoMarshaler: gogojsonpb.Marshaler{},
gogoUnmarshaler: gogojsonpb.Unmarshaler{},
protoMarshalOptions: protojson.MarshalOptions{},
protoUnmarshalOptions: protojson.UnmarshalOptions{},
gogoMarshaler: gogojsonpb.Marshaler{},
gogoUnmarshaler: gogojsonpb.Unmarshaler{},
protoMarshalOptions: protojson.MarshalOptions{},
protoUnmarshalOptions: protojson.UnmarshalOptions{},
temporalProtoUnmarshalOptions: temporalproto.CustomJSONUnmarshalOptions{},
}
}

Expand All @@ -99,6 +106,9 @@ func NewProtoJSONPayloadConverterWithOptions(options ProtoJSONPayloadConverterOp
protoUnmarshalOptions: protojson.UnmarshalOptions{
DiscardUnknown: options.AllowUnknownFields,
},
temporalProtoUnmarshalOptions: temporalproto.CustomJSONUnmarshalOptions{
DiscardUnknown: options.AllowUnknownFields,
},
options: options,
}
}
Expand Down Expand Up @@ -193,7 +203,11 @@ func (c *ProtoJSONPayloadConverter) FromPayload(payload *commonpb.Payload, value

var err error
if isProtoMessage {
err = c.protoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
if c.options.LegacyTemporalProtoCompat {
err = c.temporalProtoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
} else {
err = c.protoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
}
} else if isGogoProtoMessage {
err = c.gogoUnmarshaler.Unmarshal(bytes.NewReader(payload.GetData()), gogoProtoMessage)
}
Expand Down
Loading
Loading