Protocol buffers complier protoc
generated structures based on message
definition in *.proto
file. It's possible to use these generated structures
directly, but it's better to have clear separation between transport level
(gRPC) and business logic with its own structures. In this case you have to
convert protobuf structures into business logic structures and vice versa.
protoc-gen-struct-transformer
is a plugin for protoc
which generates functions
for structure transformation.
Let's look at simple example.
Source proto file:
// message.proto
syntax = "proto3";
package messages;
message Product {
int32 id = 1;
string name = 2;
}
Command protoc --gogofaster_out=. message.proto
will generate message.pb.go
with
following structure:
type Product struct {
Id int32 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,json=name,proto3" json:"name,omitempty"`
}
and let's suppose you service has as repo
package with ProductModel
struct inside:
type ProductModel struct {
ID int `db:"id" json:"id"`
Name string `db:"name" json:"name"`
}
In order to publish data from the repo to API you have to convert ProductModel
to Product
, for saving data which hit API you have to make back transformation.
func ProductModelToProduct(m repo.ProductModel) proto.Product {
return proto.Product {
Id: m.ID,
Name: m.Name,
}
}
func ProductToProductModel(p proto.Product) repo.ProductModel {
return repo.ProductModel {
ID: p.Id,
Name: p.Name,
}
}
List of function type should be generated:
Source | Destination | Suffix name |
---|---|---|
*proto | *model | Ptr |
[]*proto | []*model | PtrList |
*proto | model | PtrVal |
[]*proto | []model | PtrValList |
proto | model | |
proto | *model | ValPtr |
[]proto | []model | ValList |
*model | *proto | Ptr |
[]*model | []*proto | PtrList |
*model | proto | PtrVal |
[]*model | []proto | PtrValList |
model | proto | |
model | *proto | ValPtr |
[]model | []proto | ValList |
function name has a format <Source>To<Destination><Suffix>
.
For instance, function which converts list of pointers to Product into list of
Product values will be named PbToProductPtrValList
,
where
Pb
is a replacement for proto messageProducts
is a name of model structurePtrValList
is a suffix pointed that convertion is made from slice of pointer to slice of values.
Full set of function for Product message will be as:
func PbToProductPtr(src *example.Product, opts ...TransformParam) *model.Product
func PbToProductPtrList(src []*example.Product, opts ...TransformParam) []*model.Product
func PbToProductPtrVal(src *example.Product, opts ...TransformParam) model.Product
func PbToProductPtrValList(src []*example.Product, opts ...TransformParam) []model.Product
func PbToProductList(src []*example.Product, opts ...TransformParam) []model.Product
func PbToProduct(src example.Product, opts ...TransformParam) model.Product
func PbToProductValPtr(src example.Product, opts ...TransformParam) *model.Product
func PbToProductValList(src []example.Product, opts ...TransformParam) []model.Product
func ProductToPbPtr(src *model.Product, opts ...TransformParam) *example.Product
func ProductToPbPtrList(src []*model.Product, opts ...TransformParam) []*example.Product
func ProductToPbPtrVal(src *model.Product, opts ...TransformParam) example.Product
func ProductToPbValPtrList(src []model.Product, opts ...TransformParam) []*example.Product
func ProductToPbList(src []model.Product, opts ...TransformParam) []*example.Product
func ProductToPb(src model.Product, opts ...TransformParam) example.Product
func ProductToPbValPtr(src model.Product, opts ...TransformParam) *example.Product
func ProductToPbValList(src []model.Product, opts ...TransformParam) []example.Product
where
example
is a package generated byprotoc-gen-go
orprotoc-gen-gogo
pluginmodel
is a package which contains manually created models structures.
Full example you can find in example directory.
I assume you already have protoc
installed.
First of all, it's necessary to install plugin itself, it's just a binary file,
which should be placed into $PATH to be available for protoc
.
% brew tap bold-commerce/tap
% brew install protoc-gen-struct-transformer
If you're going to make changes to plugin, use go get ...
or git clone ...
% export GO111MODULE=on
% go get -u -d github.com/bold-commerce/protoc-gen-struct-transformer
% cd $GOPATH/src/github.com/bold-commerce/protoc-gen-struct-transformer
// make changes
% go install
Next, we need protoc-gen-go
plugin (or protoc-gen-gogofaster
if you use
gogo
specific options) which creates *.pb.go
file.
go get -u github.com/golang/protobuf/protoc-gen-go
// or
go get -u github.com/gogo/protobuf/protoc-gen-gogofaster
To configure plugin you have to use file level options listed below. The plugin will not process file without these options.
// This import allows to use this options.
// Relatively to import_path: github.com/bold-commerce/protoc-gen-struct-transformer.
// See Makefile for mapping details.
import "options/annotations.proto";
// Go package name which contains business logic structures.
option (transformer.go_repo_package) = "models";
// Go package name with protobuf generated srtuctures. Could be equal to
// options go_package.
option (transformer.go_protobuf_package) = "example";
// Path to source file with Go structures which will be used as destination.
option (transformer.go_models_file_path) = "example/model/model.go";
as well as message level option
// Name of structure from business logic package. This option links business
// logic and generated structure.
message Product {
option (transformer.go_struct) = "ProductModel";
// ...
}
options above are minimal requirement for use this plugin.
Also plugin has additional field level options:
message Product {
// SomeField will not be added to transformation function.
string some_field = 4 [ (transformer.skip) = true ];
// "map_as" option is used in cases when protoc-gen-go* plugin creates
// "unpredictable" field name, i.e. by default protoc-gen-go* converts
// protobuf named writen in snake_case into CamelCase, but for fields like
// map_field_1 this rule has aa exception, in pb.go file it will be
// "MapField_1" instead of "MapField1".
// "map_to" options is used when you need to map current message field to
// field in model with arbitrary name.
// Both options "map_as" and "map_to" can be used independently.
string map_field_1 = 6 [ (transformer.map_as) = "MapField_1", (transformer.map_to) = "MapField1"];
// "custom" allows to use custom transformers for fields, which require extended transformation
// The plugin won't generate methods for this field,
// but rather expect it to be in the same package with the transformer file
CustomType custom_field [(transformer.custom) = true]
}
protoc \
--proto_path=github.com/gogo:. \
--go_out=Moptions/annotations.proto=github.com/bold-commerce/protoc-gen-struct-transformer/options,plugins=grpc:. \
--struct-transformer_out=package=transform:. \
./message.proto
this command generates two files:
message.pb.go
contains auto-generated structures.transform/message_transformer.go
contains transformation functions.
by default message_transformer.go
does not contain imports. To add imports
run protoc
with:
--struct-transformer_out=package=transform,goimports=true:. \
func (s *server) CreateProduct(ctx context.Context, req *pb.Request) (*pb.Response, error) {
p, err := s.svc.Create(ctx, transform.PbToProduct(req.Product))
if err != nil {
return nil, err
}
return &pb.Response{
Product: transform.ProductToPb(p),
}, nil
}
Usage of protoc-gen-struct-transformer:
-debug
Add debug information to generated file.
-goimports
Perform goimports on generated file.
-helper-package string
Package name for helper functions.
-package string
Package name for generated functions. (default "fallback")
-use-package-in-path
If true, package parameter will be used in path for output file. (default true)
-version
Print current version.
gogo.proto
file which is used for gogo-specific options is imported from
go modules cache. In order to fill out the cache run:
% export GO111MODULE=on
% go build
and run make generate
again.