β€οΈ PowerProto is actively maintained! Any questions in use can be directly raised issue, I will respond to you as fast as possible. If you think the project is helpful to you, please give me a star to encourage me!
English | δΈζ
PowerProto is used to solve the following three main problems:
- lower the usage threshold and usage cost of gRPC.
- solve the version control problem of protoc and its related plugins (such as protoc-gen-go, protoc-gen-grpc-gateway).
- efficiently manage the compilation of proto to achieve multi-platform compatibility, one-click installation and compilation.
- one-click installation and multi-version management of protoc.
- one-click installation and multi-version management of protoc related plugins (such as protoc-gen-go).
- manage the compilation of proto through config file instead of shell script to improve readability and compatibility.
- bootstrap generation of config files, cross-platform compatibility, a config can be compiled in multiple platforms with one click.
- support batch and recursive compilation of proto files to improve efficiency.
- cross-platform support PostAction, you can perform some routine operations (such as replacing "omitempty" in all generated files) after the compilation.
- support PostShell, execute specific shell scripts after the compilation.
- one-click installation and version control of google apis and gogo protobuf etcγ
- The current version of
PowerProto
relies ongo
(>=1.16) andgit
(in the future it may use CDN to pull built binaries directly), please make sure the runtime environment contains these two commands. protoc
download source is Github,PowerProto
respectsHTTP_PROXY
,HTTPS_PROXY
environment variables when downloadingprotoc
, if you encounter network problems, please configure your own proxy.- When querying the version list of
protoc
,git ls-remote
is used forgit.luolix.top
, if you encounter network problems, please configure the proxy forgit
by yourself. - In the current version, downloading and querying plugin versions rely on the
go
command, so if you encounter network problems, please configure theGOPROXY
environment variable yourself. - By default,
user directory/.powerproto
is used as the installation directory, which is used to place the downloaded plug-ins and global config. - If you think the name
powerproto
is too long, you canalias
it into a simpler name to improve the input efficiency, for example, no one will mind if you call itpp
.
Installation can be performed by executing the following command directly:
go install github.com/storyicon/powerproto/cmd/powerproto@latest
You can download the out-of-the-box version via the Github Release Page
You can view help with powerproto -h
, e.g.
powerproto -h
powerproto init -h
powerproto tidy -h
powerproto build -h
powerproto env -h
It has the advantage that the documentation on the command line is always consistent with your binary version.
The config can be initialized with the following command.
powerproto init
The config can be tidied with the following command.
powerproto tidy
It will search for a config file named powerproto.yaml
from the current directory to the parent directory, and will read and tidy the config.
You can also specify which config file to tidy.
powerproto tidy [the path of proto file]
Tidy the config consists of two main operations:
- replacing the latest in the version with the real latest version number by querying.
- install all dependencies defined in the config file.
Supports entering debug mode
by appending the -d
argument to see more detailed logs.
The Proto file can be compiled with the following command.
// Compile the specified proto file
powerproto build xxxx.proto
// Compile all the proto files in the current directory
powerproto build .
// Compile all proto files in the current directory recursively, including subfolders.
powerproto build -r .
The execution logic is that for each proto file, the powerproto.yaml
config file will be searched from the directory where the proto file is located to the ancestor directory:
- For the found config file, match it with the
scope
in it and use it if it matches. - Check and install the dependencies declared in the config file.
- Compile the proto file according to the
plugins
,protoc
,options
,importPaths
and other configs in the config fileγ After all the proto files are compiled, if you specify the-p
argument,PostAction
andPostShell
will also be executed.
Note: The default working directory
of PowerProto
is the directory where the proto file
matches to the config file, it is equivalent to the directory where you execute the protoc
command. You can change it via protocWorkDir
in the config file.
Supports entering debug mode
by appending the -d
argument to see more detailed logs.
Supports entering dryRun mode
by appending the -y
argument, in this mode the commands are not actually executed, but just printed out, which is very useful for debugging.
If your command keeps getting stuck in a certain state, there is a high probability that there is a network problem.
You can check if the environment variables are configured successfully with the following command:
powerproto env
For example, you have the following file structure in the /mnt/data/hello
directory:
$ pwd
/mnt/data/hello
$ tree
./apis
βββ hello.proto
powerproto.yaml
The contents of the powerproto.yaml
file (you can easily generate the config file with the powerproto init
command) are:
scopes:
- ./
protoc: latest
protocWorkDir: ""
plugins:
protoc-gen-go: google.golang.org/protobuf/cmd/protoc-gen-go@latest
protoc-gen-go-grpc: google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
repositories:
GOOGLE_APIS: https://github.com/googleapis/googleapis@75e9812478607db997376ccea247dd6928f70f45
options:
- --go_out=.
- --go_opt=paths=source_relative
- --go-grpc_out=.
- --go-grpc_opt=paths=source_relative
importPaths:
- .
- $GOPATH
- $POWERPROTO_INCLUDE
- $GOOGLE_APIS/github.com/googleapis/googleapis
postActions: []
postShell: ""
Execute in any directory:
powerproto build -r /mnt/data/hello/apis
You can get the compiled file:
$ pwd
/mnt/data/hello
$ tree
./apis
βββ hello.pb.go
βββ hello.proto
βββ hello_grpc.pb.go
powerproto.yaml
It is equivalent to if you were in the directory where powerproto.yaml
is located and executed:
$POWERPROTO_HOME/protoc/3.17.3/protoc --go_out=. \
--go_opt=paths=source_relative \
--go-grpc_out=. \
--go-grpc_opt=paths=source_relative \
--proto_path=/mnt/data/hello \
--proto_path=$GOPATH \
--proto_path=$POWERPROTO_HOME/include \
--proto_path=$POWERPROTO_HOME/gits/75e9812478607db997376ccea247dd6928f70f45/github.com/googleapis/googleapis \
--plugin=protoc-gen-go=$POWERPROTO_HOME/plugins/google.golang.org/protobuf/cmd/protoc-gen-go@v1.27.1/protoc-gen-go \
--plugin=protoc-gen-go-grpc=$POWERPROTO_HOME/plugins/google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1.0/protoc-gen-go-grpc \
/mnt/data/hello/apis/hello.proto
More examples can be found in examples.
The config file is used to describe the versions of various dependencies and parameters when compiling the proto file.
It can be easily initialized with powerproto init
.
Take the following config file as an example:
# required. scopes is used to define scopes.
# i.e. which directories in the project the current config item is valid for
scopes:
- ./
# required. the version of protoc.
# you can fill in the 'latest', will be automatically converted to the latest version
protoc: 3.17.3
# optional. The working directory for executing the protoc command,
# the default is the directory where the config file is located.
# support mixed environment variables in path, such as $GOPATH
protocWorkDir: ""
# optional. define dependent Git repositories
# Generally used for dependency control of public protobuf libraries
repositories:
# Definition depends on the 27156597fdf4fb77004434d4409154a230dc9a32 version of https://github.com/googleapis/googleapis
# and defines its name as GOOGLE_APIS
# It can be referenced in importPaths by $GOOGLE_APIS
GOOGLE_APIS: https://github.com/googleapis/googleapis@27156597fdf4fb77004434d4409154a230dc9a32
# Definition depends on the 226206f39bd7276e88ec684ea0028c18ec2c91ae version of https://github.com/gogo/protobuf
# and defines its name as GOGO_PROTOBUF
# It can be referenced in the importPaths by $GOGO_PROTOBUF
GOGO_PROTOBUF: https://github.com/gogo/protobuf@226206f39bd7276e88ec684ea0028c18ec2c91ae
# required. it is used to describe which plug-ins are required for compilation
plugins:
# the name, path, and version number of the plugin.
# the address of the plugin must be in path@version format,
# and version can be filled with 'latest', which will be automatically converted to the latest version.
protoc-gen-deepcopy: istio.io/tools/cmd/protoc-gen-deepcopy@latest
protoc-gen-go: google.golang.org/protobuf/cmd/protoc-gen-go@latest
protoc-gen-go-json: github.com/mitchellh/protoc-gen-go-json@v1.0.0
protoc-gen-grpc-gateway: github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@v2.5.0
# required. defines the parameters of protoc when compiling proto files
# In options, you can still use variables like $GOPATH, $SOURCE_RELATIVE, $GOGO_PROTOBUF as in importPaths
options:
- --go_out=paths=source_relative:.
- --go-json_out=.
- --deepcopy_out=source_relative:.
- --grpc-gateway_out=.
- --go-grpc_out=paths=source_relative:.
# required. defines the path of the proto dependency, which will be converted to the --proto_path (-I) parameter.
importPaths:
# Special variables. Will be replaced with the folder where the current configuration file is located.
- .
# Environment variables. Environment variables can be used in importPaths.
# Support mixed writing like $GOPATH/include
- $GOPATH
# Special variables. Will be replaced with the local path to the public proto file that comes with protoc by default
- $POWERPROTO_INCLUDE
# Special variables. Reference to the directory where the proto file to be compiled is located
# For example, if /a/b/data.proto is to be compiled, then the /a/b directory will be automatically referenced
- $SOURCE_RELATIVE
# References GOOGLE_APIS as defined in repositories
- $GOOGLE_APIS/github.com/googleapis/googleapis
# References GOGO_PROTOBUF as defined in repositories
- $GOGO_PROTOBUF
# optional. The operation is executed after compilation.
# its working directory is the directory where the config file is located.
# postActions is cross-platform compatible.
# Note that the "-p" parameter must be appended to the "powerproto build" to allow execution of the postActions in the config file
postActions: []
# optional. The shell script that is executed after compilation.
# its working directory is the directory where the config file is located.
# postShell is not cross-platform compatible.
# Note that the "-p" parameter must be appended to the "powerproto build" to allow execution of the postShell in the config file
postShell: |
// do something
When building the proto file, the powerproto.yaml
config file will be searched from the directory where the proto file is located to the ancestor directory, match with the scope
in.
The first matched config item will be used for the compilation of this proto file.
When PowerProto executes protoc (and also when it executes postActions and postShell), the default is to use the directory where the config file is located as the working directory. (working directory is equivalent to the directory where you execute the protoc command.)
A config file can be filled with multiple configs, which are separated by "---".
In the example below, the apis1 directory uses protoc-gen-go with v1.25.0, while the apis2 directory uses protoc-gen-go with v1.27.0.
scopes:
- ./apis1
protoc: v3.17.3
protocWorkDir: ""
plugins:
protoc-gen-go: google.golang.org/protobuf/cmd/protoc-gen-go@v1.25.0
protoc-gen-go-grpc: google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1.0
repositories:
GOOGLE_APIS: https://github.com/googleapis/googleapis@75e9812478607db997376ccea247dd6928f70f45
options:
- --go_out=.
- --go_opt=paths=source_relative
- --go-grpc_out=.
- --go-grpc_opt=paths=source_relative
importPaths:
- .
- $GOPATH
- $POWERPROTO_INCLUDE
- $GOOGLE_APIS/github.com/googleapis/googleapis
postActions: []
postShell: ""
---
scopes:
- ./apis2
protoc: v3.17.3
protocWorkDir: ""
plugins:
protoc-gen-go: google.golang.org/protobuf/cmd/protoc-gen-go@v1.27.0
protoc-gen-go-grpc: google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1.0
repositories:
GOOGLE_APIS: https://github.com/googleapis/googleapis@75e9812478607db997376ccea247dd6928f70f45
options:
- --go_out=.
- --go_opt=paths=source_relative
- --go-grpc_out=.
- --go-grpc_opt=paths=source_relative
importPaths:
- .
- $GOPATH
- $POWERPROTO_INCLUDE
- $GOOGLE_APIS/github.com/googleapis/googleapis
postActions: []
postShell: ""
PostAction allows to perform specific actions after all proto files have been compiled. In contrast to PostShell
, it is cross-platform supported.
For security reasons, the PostAction
and PostShell
defined in the config file will only be executed if the -p
argument is appended to the execution of powerproto build
.
Currently, PostAction supports the following commands:
Command | Description | Function Prototype |
---|---|---|
copy | Copy file or folder | copy(src string, dest string) error |
move | Move file or folder | move(src string, dest string) error |
remove | Delete file or folder | remove(path ...string) error |
replace | Batch replacement of strings in files | replace(pattern string, from string, to string) error |
For copying files or folders, the function prototype is:
copy(src string, dest string) error
For security and config compatibility, only relative paths are allowed in the parameters.
If the target folder already exists, it will be merged.
The following example will copy 'a' from the directory where the config file is located to 'b'.
postActions:
- name: copy
args:
- ./a
- ./b
For moving files or folders, the function prototype is:
move(src string, dest string) error
For security and config compatibility, only relative paths are allowed in the parameters.
If the target folder already exists, it will be merged.
The following example will move 'a' in the directory where the config file is located to 'b'.
postActions:
- name: move
args:
- ./a
- ./b
For deleting files or folders, the function prototype is:
remove(path ...string) error
For security and config compatibility, only relative paths are allowed in the parameters.
The following example will remove 'a', 'b' and 'c' from the directory where the config file is located:
postActions:
- name: remove
args:
- ./a
- ./b
- ./c
Used for batch replacement of strings in files. Its function prototype is:
replace(pattern string, from string, to string) error
pattern
is a relative path that supports wildcard characters.from
is the string to be replaced.to
is the string to replace with.
The following example will replace ,omitempty
with the empty string in all go files in the apis directory and its subdirectories:
postActions:
- name: replace
args:
- ./apis/**/*.go
- ',omitempty'
- ""