[ENG-447] protobuf workspaces migration

[adisaran64]

Description

• migrating protobuf code generator to protobuf workspaces
• ticket: ENG-447
• purpose: current proto build configuration for proto and swagger files "not reliable" (as per ticket), this PR creates a Buf Workspace in the repository and changes the iterative use of protoc -> buf build and buf generate.

---

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

PR review checkboxes:

I have...

• added a relevant changelog entry to the Unreleased section in CHANGELOG.md
• included the correct type prefix in the PR title
• targeted the correct branch (see PR Targeting)
• provided a link in the PR description to the relevant issue or specification
• reviewed "Files changed" and left comments if necessary
• confirmed all required CI checks have passed

Code maintenance:

I have...

• written unit and integration tests
• added relevant godoc and code comments.
• updated relevant documentation (docs/) or specification (x/<module>/spec/)

---

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed all author checklist items have been addressed
• confirmed that this PR does not change production code

[fedekunze]

looks like make proto-gen is failing. Can you look into it?

[adisaran64]

I think changing the plugin from swagger to openapi is a much more involved task (not just renaming), would involve moving from protoc-gen-swagger to protoc-gen-api, swagger-combine and all swagger related tools throughout the repository would have to all be removed and updated. can change the name of the temp folder if you want, but for now will leave it matching with the plugin name.

[adisaran64]

with current commit, ran make proto-gen locally with the docker container, it generates the appropriate .proto files.

[fedekunze]

I think changing the plugin from swagger to openapi is a much more involved task (not just renaming), would involve moving from protoc-gen-swagger to protoc-gen-api, swagger-combine and all swagger related tools throughout the repository would have to all be removed and updated. can change the name of the temp folder if you want, but for now will leave it matching with the plugin name.

Sounds good. Let's create an issue for the follow up tasks

[fedekunze]

with current commit, ran make proto-gen locally with the docker container, it generates the appropriate .proto files.

tried running it but I'm getting the following error

Failed to "build": File "proto/buf.yaml" has unknown configuration version: v1.
make: *** [proto-gen] Error 1

[adisaran64]

with current commit, ran make proto-gen locally with the docker container, it generates the appropriate .proto files.

tried running it but I'm getting the following error

Failed to "build": File "proto/buf.yaml" has unknown configuration version: v1.
make: *** [proto-gen] Error 1

not sure why this is the case for you, works fine on my end, and as far as I'm aware v1 is the only valid configuration version for the .yaml files in a buf workspace. could you please let me know the specific commands that you are using so I can try to reproduce the error? @fedekunze

[fedekunze]

@danburck @ramacarlucho can you test from your end too?

[danburck]

@adisaran64 can you

• add in the description the effects of this change,
• link a linear ticket if there is one and
• add a changelog?

@adisaran64 @fedekunze make proto-gen is failing for me

➜ make proto-gen
Generating Protobuf files
/usr/local/bin/docker run --rm -v /Users/danburck/code/evmos/evmos:/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen.sh
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.
./scripts/protocgen.sh: line 54: statik: not found
make: *** [proto-gen] Error 127

[adisaran64]

@danburck added those elements to the PR/changelog.
Realized that statik was already on my docker image but not installed through the script, added a line in protoc-gen.sh (run by make proto-gen to get statik (although this isn't directly related to the generation of the .pb.go files). Would you mind trying it once more?

[adisaran64]

@danburck added build to workflow to generate code files when .proto files change

[danburck]

Nice @adisaran64, I ran make proto-gen again. Can you have a look at the following things:

• Looks like the docs/protocol/proto-docs.md only includes ibc core messages after the execution. Is that expected?

* Getting a bunch of Warnings `Warning: Duplicate generated file name "proto-docs.md". Buf will continue without error here and drop the second occurrence of this file, but please raise an issue with the maintainer of the plugin.`

Other than that the proto generation works fine 👍

[adisaran64]

@danburck for some reason the warnings were suppressed for me, didn't notice the docs generation had issues. thanks for letting me know.

turns out the docs/protocol/proto-docs.md file was being overwritten multiple times when generating swagger documentation (the second set of buf generate calls in the loop in the scripts/protocgen.sh file). split up the buf.gen.yaml file into two files so that it's not overwritten, changed the buf strategy so that all files are collected before generating docs (so not only a single subdirectory of proto is documented).

file generated correctly now, you should be able to confirm by removing your docs/protocol/proto-docs.md file and running make proto-gen. the warnings shouldn't appear but it would be good to check, since I couldn't see them before or after.

[danburck]

ACK Great work @adisaran64 . Tested and works fine. Note, that I pushed a commit that fixes a deprecation warning.