Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for CometBFT proto 0.38 #1312

Merged
merged 37 commits into from
May 19, 2023
Merged

Support for CometBFT proto 0.38 #1312

merged 37 commits into from
May 19, 2023

Conversation

mzabaluev
Copy link
Contributor

@mzabaluev mzabaluev commented May 2, 2023

Protobuf-related changes for #1301

  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Added entry in .changelog/

Copy link
Contributor Author

@mzabaluev mzabaluev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raising open issues to address for completing this PR.

abci/src/application.rs Show resolved Hide resolved
abci/src/application/kvstore.rs Outdated Show resolved Hide resolved
abci/tests/kvstore_app.rs Outdated Show resolved Hide resolved
mzabaluev added 25 commits May 5, 2023 13:59
Check out the protos from the v0.38.0-alpha.1 release tag and
add the v0_38 fork of generated modules to tendermint-proto.
The AbciParams message/object was added in 0.38, used as the
"abci" field in ConsensusParams.
Add protobuf conversions for v0_38 protos, and serde.
This is needed for the more detailed VoteInfo in 0.38.
The BlockSignatureInfo enum is an attempt to represent both
the pre-0.38 encoding with the signed_last_block flag, and the
current one where the signature information is given by the
block_id_flag field.
Rename to Signature::into_bytes to follow Rust naming conventions.
The serialization of extension fields is not tested yet.
Change the tendermint-proto structs in use to v0_38
and update the application API accordingly.
Only change enough to compile, will likely not work.
As commented in the declaration of the CheckTx domain type,
I'm not sure it's the right thing to do.
There was no apparent value added by defining the impls manually,
IDK why the attributes were previously shifted to the enum type.
Add members to VoteInfo to be able to convert from and to
the ExtendedVoteInfo message type as defined in 0.38 proto.
Define conversions from and to the proto type, as well as
conversions between CommitInfo and ExtendedCommitInfo in the
proto.

As a helper to the above, provide Bytes conversions for the
Signature domain type.
Add the v0_38::abci::{Request,Response} enums and the new domain types
representing request and response messages added in CometBFT 0.38.
@mzabaluev mzabaluev force-pushed the mikhail/cometbft-0.38 branch from 6d755a2 to 36eca7a Compare May 5, 2023 11:31
mzabaluev added 2 commits May 8, 2023 14:27
Define {ConsensusRequest, InfoRequest, MempoolRequest, SnapshotRequest}
separately for each of the v0_*::abci versioned modules, enumerating
just the requests that can be used with the respective version.
No panic.
@codecov-commenter
Copy link

codecov-commenter commented May 9, 2023

Codecov Report

Merging #1312 (270148a) into main (eb7e757) will decrease coverage by 2.3%.
The diff coverage is 11.5%.

❗ Current head 270148a differs from pull request most recent head 1f5688d. Consider uploading reports for the commit 1f5688d to get more accurate results

@@           Coverage Diff           @@
##            main   #1312     +/-   ##
=======================================
- Coverage   60.3%   58.1%   -2.3%     
=======================================
  Files        275     281      +6     
  Lines      24420   25467   +1047     
=======================================
+ Hits       14733   14802     +69     
- Misses      9687   10665    +978     
Impacted Files Coverage Δ
abci/src/codec.rs 89.1% <ø> (ø)
abci/src/error.rs 0.0% <ø> (ø)
abci/tests/echo_app.rs 100.0% <ø> (ø)
p2p/src/secret_connection.rs 86.8% <ø> (ø)
p2p/src/secret_connection/amino_types.rs 0.0% <ø> (ø)
p2p/src/secret_connection/protocol.rs 59.8% <ø> (ø)
proto/src/serializers/evidence.rs 17.9% <0.0%> (-9.0%) ⬇️
proto/tests/unit.rs 94.8% <ø> (ø)
tendermint/src/abci/event.rs 54.5% <0.0%> (-7.2%) ⬇️
tendermint/src/abci/request/begin_block.rs 0.0% <0.0%> (ø)
... and 44 more

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

mzabaluev added 2 commits May 9, 2023 16:46
Just like with abci::Request and abci::Response, the categorized
request and response enums get re-exported from the v0_38::abci
module.
@mzabaluev mzabaluev marked this pull request as ready for review May 9, 2023 19:25
The ExtendedVoteInfo and ExtendedCommitInfo protobuf messages
can be encoded only in RequestPrepareProposal. In other contexts
the extension fields are not useful.
Comment on lines 53 to 62
pub enum BlockSignatureInfo {
/// Full information available, as determined by the `BlockIdFlag` value.
Flag(block::BlockIdFlag),
/// In CometBFT versions before 0.38, the `signed_last_block` field has
/// the value of true.
///
/// This variant should not be used in CometBFT 0.38 or later
/// and will result in the "undefined" encoding in protobuf.
LegacySigned,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sergio-mena This is my attempt to represent the block signing information in VoteInfo and ExtendedVoteInfo without data loss also when converted from older versions.
Something to consider in connection with cometbft/cometbft#780

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a problem here. How about the case where we have the "legacy" field set to false?
If these data structures are to work for, e.g., legacy-to-legacy interoperability, you are not able to handle the case where the signature is absent.

Copy link
Contributor

@thanethomson thanethomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor comments, but otherwise this looks great. Nice work @mzabaluev!

abci/src/application.rs Show resolved Hide resolved
tendermint/src/consensus/params.rs Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
tendermint/src/vote.rs Outdated Show resolved Hide resolved
tendermint/src/abci/response/check_tx.rs Outdated Show resolved Hide resolved
rpc/src/dialect/check_tx.rs Outdated Show resolved Hide resolved
tendermint/src/abci/types.rs Outdated Show resolved Hide resolved
@mzabaluev mzabaluev changed the title Support for CometBFT 0.38 Support for CometBFT proto 0.38 May 12, 2023
mzabaluev added 4 commits May 12, 2023 14:22
As suggested by Thane.
Add versioning information to comments on the fields of domain types
that have been added in 0.38.
Copy link
Contributor

@thanethomson thanethomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I'd be interested to know what you think, @hdevalence and @romac, if you have time to review.

Comment on lines 6 to 8
// FIXME: do we want to support this for older versions of the protocol?
// Does anybody use it?
//pub data: Bytes,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also restoring this field for the sake of pre-0.38 applications.

@mzabaluev mzabaluev merged commit bdef988 into main May 19, 2023
@mzabaluev mzabaluev deleted the mikhail/cometbft-0.38 branch May 19, 2023 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants