Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for CometBFT proto 0.38 #1312

Merged
merged 37 commits into from
May 19, 2023
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
3f2dfb2
Generate protos for v0_38
mzabaluev Apr 24, 2023
eba4066
proto: add serializers for v0_38::types::Evidence
mzabaluev Apr 24, 2023
b5273a1
p2p: switch tendermint-proto imports to v0_38
mzabaluev Apr 24, 2023
7afb61f
tendermint: domain type for AbciParams
mzabaluev Apr 25, 2023
71b833b
tendermint: BlockIdFlag as a domain type
mzabaluev Apr 27, 2023
6f51d23
Extend VoteInfo to support 0.38 data
mzabaluev Apr 27, 2023
ad1f3f0
Rename Signature::to_bytes
mzabaluev Apr 27, 2023
10ac41b
tendermint: v0_38 serialization for Vote
mzabaluev Apr 27, 2023
4d6247d
proto: use v0_38 in BlockId unit tests
mzabaluev Apr 27, 2023
039a773
tendermint: v0_38 serialization for block::Size
mzabaluev Apr 27, 2023
bfdc346
tendermint: v0_38 conversions for abci::Event
mzabaluev Apr 28, 2023
df3abf0
Extend tendermint_pb_modules to v0_38
mzabaluev Apr 27, 2023
b063c47
abci: Update to ABCI 0.38
mzabaluev Apr 27, 2023
0176408
abci: temporarily fudge tests for proto 0_38
mzabaluev May 2, 2023
2601aac
light-client-verifier: adapt to new Vote fields
mzabaluev May 2, 2023
87fd393
rpc: ignore legacy CheckTx fields
mzabaluev May 2, 2023
06703cc
test: fix proto import in p2p/secret_connection
mzabaluev May 2, 2023
913a902
testgen: default the new fields for 0.38
mzabaluev May 2, 2023
1a41d12
CI: update buf and protoc to latest versions
mzabaluev May 2, 2023
82bba88
CI: diff between generated and checked-in files
mzabaluev May 3, 2023
3619b7e
Restore serde derives on PublicKey
mzabaluev May 3, 2023
a060aa3
Regenerate protos from latest release tags
mzabaluev May 2, 2023
9618fd1
tendermint: v0_38 proto support for request::Info
mzabaluev May 4, 2023
431db0f
tendermint: support 0.38 extensions for VoteInfo
mzabaluev May 4, 2023
36eca7a
tendermint: v0.38 ABCI requests and responses
mzabaluev May 4, 2023
b4e7779
Fix redundant closure lints
mzabaluev May 5, 2023
c08249f
tendermint: version-specific kinded response enums
mzabaluev May 8, 2023
f169a93
abci: crude logic and tests for finalize_block
mzabaluev May 9, 2023
0da192b
tendermint: Re-export v0_38 kinded ABCI enums
mzabaluev May 9, 2023
58b4dc5
Changelog entries for #1312
mzabaluev May 9, 2023
fe0c542
tendermint: domain type for ExtendedVoteInfo
mzabaluev May 10, 2023
43ebace
Fix formatting of changelog entries
mzabaluev May 12, 2023
6a19f6c
tendermint: 0.38 version notes on domain types
mzabaluev May 12, 2023
ea7cd5f
Update buf to 1.18 in CI
mzabaluev May 12, 2023
1f5688d
Restore legacy fields on CheckTx
mzabaluev May 12, 2023
3914320
tendermint: restore data field in Commit response
mzabaluev May 15, 2023
483a1da
Update changelog for #1312
mzabaluev May 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .changelog/unreleased/breaking-changes/1312-cometbft-0.38.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
- [`tendermint`] Adaptations for CometFBT 0.38
mzabaluev marked this conversation as resolved.
Show resolved Hide resolved
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312)):
* Define `consensus::params::AbciParams` struct, add the `abci` field of this
type to `consensus::Params` to represent the protobuf additions.
* Represent vote extensions in the `Vote` domain type.
* Remove the fields from the `CheckTx` and `Commit` domain types that are
no longer present in CometBFT 0.38.
- [`tendermint`] Define version-specific categorized request/response enums:
`ConsensusRequest`, `MempoolRequest`, `InfoRequest`, `ShapshotRequest`,
`ConsensusResponse`, `MempoolResponse`, `InfoResponse`, `ShapshotResponse`,
in each of the `v0_*::abci` modules, so that the variants are trimmed to the
requests/responses used by the respective protocol version.
Reexport the types from `v0_38::abci` as continuation for these names in the
`abci` module
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312)).
- [`tendermint`] Rename `Signature::to_bytes` to `Signature::into_bytes`
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312)).
- [`tendermint-abci`] Update the `Application` interface to CometBFT 0.38
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312))
12 changes: 12 additions & 0 deletions .changelog/unreleased/improvements/1312-cometbft-0.38.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
- [`tendermint-proto`] Generate prost bindings for CometBFT 0.38
under the `tendermint::v0_38` module
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312))
- [`tendermint`] Support for CometBFT 0.38:
([\#1312](https://github.com/informalsystems/tendermint-rs/pull/1312)):
* Add conversions to and from `tendermint::v0_38` protobuf
types generated in [`tendermint-proto`].
* Add request and response enums under `v0_38::abci` to enumerate all requests
and responses appropriate for CometBFT version 0.38.
* Add request and response types under `abci` to represent the requests
and responses new to ABCI++ 2.0 in CometBFT version 0.38. The names are
`ExtendVote`, `FinalizeBlock`, `VerifyVoteExtension`.
6 changes: 4 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,18 +69,20 @@ jobs:
override: true
- name: Install protoc
run: |
curl -Lo /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v21.4/protoc-21.4-linux-x86_64.zip
curl -Lo /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v22.3/protoc-22.3-linux-x86_64.zip
unzip /tmp/protoc.zip -d ${HOME}/.local
echo "PROTOC=${HOME}/.local/bin/protoc" >> $GITHUB_ENV
export PATH="${PATH}:${HOME}/.local/bin"
- name: Install buf
run: |
curl -sSL https://github.com/bufbuild/buf/releases/download/v1.15.1/buf-Linux-x86_64 \
curl -sSL https://github.com/bufbuild/buf/releases/download/v1.17.0/buf-Linux-x86_64 \
mzabaluev marked this conversation as resolved.
Show resolved Hide resolved
-o /usr/local/bin/buf
chmod +x /usr/local/bin/buf
- name: Regenerate proto definitions
working-directory: ./tools/proto-compiler/
run: cargo run
- name: Show the differences with checked-in files
run: git diff -- proto/src/prost
- name: Ensure that generated proto definitions compile
uses: actions-rs/cargo@v1
with:
Expand Down
2 changes: 1 addition & 1 deletion abci/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ description = """
[[bin]]
name = "kvstore-rs"
path = "src/application/kvstore/main.rs"
required-features = [ "binary", "kvstore-app" ]
required-features = [ "binary", "client", "kvstore-app" ]

[features]
default = ["flex-error/std", "flex-error/eyre_tracer"]
Expand Down
61 changes: 34 additions & 27 deletions abci/src/application.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@ pub mod echo;
#[cfg(feature = "kvstore-app")]
pub mod kvstore;

use tendermint_proto::v0_37::abci::{
request::Value, response, response_process_proposal, Request, RequestApplySnapshotChunk,
RequestBeginBlock, RequestCheckTx, RequestDeliverTx, RequestEcho, RequestEndBlock, RequestInfo,
RequestInitChain, RequestLoadSnapshotChunk, RequestOfferSnapshot, RequestPrepareProposal,
RequestProcessProposal, RequestQuery, Response, ResponseApplySnapshotChunk, ResponseBeginBlock,
ResponseCheckTx, ResponseCommit, ResponseDeliverTx, ResponseEcho, ResponseEndBlock,
ResponseFlush, ResponseInfo, ResponseInitChain, ResponseListSnapshots,
ResponseLoadSnapshotChunk, ResponseOfferSnapshot, ResponsePrepareProposal,
ResponseProcessProposal, ResponseQuery,
use tendermint_proto::v0_38::abci::{
request::Value, response, response_process_proposal, response_verify_vote_extension, Request,
RequestApplySnapshotChunk, RequestCheckTx, RequestEcho, RequestExtendVote,
RequestFinalizeBlock, RequestInfo, RequestInitChain, RequestLoadSnapshotChunk,
RequestOfferSnapshot, RequestPrepareProposal, RequestProcessProposal, RequestQuery,
RequestVerifyVoteExtension, Response, ResponseApplySnapshotChunk, ResponseCheckTx,
ResponseCommit, ResponseEcho, ResponseExtendVote, ResponseFinalizeBlock, ResponseFlush,
ResponseInfo, ResponseInitChain, ResponseListSnapshots, ResponseLoadSnapshotChunk,
ResponseOfferSnapshot, ResponsePrepareProposal, ResponseProcessProposal, ResponseQuery,
ResponseVerifyVoteExtension,
};

/// An ABCI application.
Expand Down Expand Up @@ -52,21 +53,6 @@ pub trait Application: Send + Clone + 'static {
Default::default()
}

/// Signals the beginning of a new block, prior to any `DeliverTx` calls.
fn begin_block(&self, _request: RequestBeginBlock) -> ResponseBeginBlock {
Default::default()
}

/// Apply a transaction to the application's state.
fn deliver_tx(&self, _request: RequestDeliverTx) -> ResponseDeliverTx {
Default::default()
}

/// Signals the end of a block.
fn end_block(&self, _request: RequestEndBlock) -> ResponseEndBlock {
Default::default()
}

/// Signals that messages queued on the client should be flushed to the server.
fn flush(&self) -> ResponseFlush {
ResponseFlush {}
Expand Down Expand Up @@ -147,6 +133,23 @@ pub trait Application: Send + Clone + 'static {
status: response_process_proposal::ProposalStatus::Accept as i32,
}
}

fn extend_vote(&self, _request: RequestExtendVote) -> ResponseExtendVote {
Default::default()
}

fn verify_vote_extension(
&self,
_request: RequestVerifyVoteExtension,
) -> ResponseVerifyVoteExtension {
ResponseVerifyVoteExtension {
status: response_verify_vote_extension::VerifyStatus::Accept as i32,
}
}

fn finalize_block(&self, _request: RequestFinalizeBlock) -> ResponseFinalizeBlock {
Default::default()
mzabaluev marked this conversation as resolved.
Show resolved Hide resolved
}
}

/// Provides a mechanism for the [`Server`] to execute incoming requests while
Expand All @@ -168,10 +171,7 @@ impl<A: Application> RequestDispatcher for A {
Value::Info(req) => response::Value::Info(self.info(req)),
Value::InitChain(req) => response::Value::InitChain(self.init_chain(req)),
Value::Query(req) => response::Value::Query(self.query(req)),
Value::BeginBlock(req) => response::Value::BeginBlock(self.begin_block(req)),
Value::CheckTx(req) => response::Value::CheckTx(self.check_tx(req)),
Value::DeliverTx(req) => response::Value::DeliverTx(self.deliver_tx(req)),
Value::EndBlock(req) => response::Value::EndBlock(self.end_block(req)),
Value::Commit(_) => response::Value::Commit(self.commit()),
Value::ListSnapshots(_) => response::Value::ListSnapshots(self.list_snapshots()),
Value::OfferSnapshot(req) => {
Expand All @@ -189,6 +189,13 @@ impl<A: Application> RequestDispatcher for A {
Value::ProcessProposal(req) => {
response::Value::ProcessProposal(self.process_proposal(req))
},
Value::ExtendVote(req) => response::Value::ExtendVote(self.extend_vote(req)),
Value::VerifyVoteExtension(req) => {
response::Value::VerifyVoteExtension(self.verify_vote_extension(req))
},
Value::FinalizeBlock(req) => {
response::Value::FinalizeBlock(self.finalize_block(req))
},
}),
}
}
Expand Down
80 changes: 39 additions & 41 deletions abci/src/application/kvstore.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ use std::{
};

use bytes::BytesMut;
use tendermint_proto::v0_37::abci::{
Event, EventAttribute, RequestCheckTx, RequestDeliverTx, RequestInfo, RequestQuery,
ResponseCheckTx, ResponseCommit, ResponseDeliverTx, ResponseInfo, ResponseQuery,
use tendermint_proto::v0_38::abci::{
Event, EventAttribute, RequestCheckTx, RequestFinalizeBlock, RequestInfo, RequestQuery,
ResponseCheckTx, ResponseCommit, ResponseFinalizeBlock, ResponseInfo, ResponseQuery,
};
use tracing::{debug, info};

Expand All @@ -20,9 +20,10 @@ use crate::{codec::MAX_VARINT_LENGTH, Application, Error};
/// store - the [`KeyValueStoreDriver`].
///
/// ## Example
/// ```rust
///
/// ```
/// use tendermint_abci::{KeyValueStoreApp, ServerBuilder, ClientBuilder};
/// use tendermint_proto::abci::{RequestEcho, RequestDeliverTx, RequestQuery};
/// use tendermint_proto::abci::{RequestEcho, RequestFinalizeBlock, RequestQuery};
///
/// // Create our key/value store application
/// let (app, driver) = KeyValueStoreApp::new();
Expand All @@ -46,8 +47,9 @@ use crate::{codec::MAX_VARINT_LENGTH, Application, Error};
///
/// // Deliver a transaction and then commit the transaction
/// client
/// .deliver_tx(RequestDeliverTx {
/// tx: "test-key=test-value".into(),
/// .finalize_block(RequestFinalizeBlock {
/// txs: vec!["test-key=test-value".into()],
/// ..Default::default()
/// })
/// .unwrap();
/// client.commit().unwrap();
Expand Down Expand Up @@ -175,27 +177,31 @@ impl Application for KeyValueStoreApp {
gas_used: 0,
events: vec![],
codespace: "".to_string(),
..Default::default()
}
}

fn deliver_tx(&self, request: RequestDeliverTx) -> ResponseDeliverTx {
let tx = std::str::from_utf8(&request.tx).unwrap();
let tx_parts = tx.split('=').collect::<Vec<&str>>();
let (key, value) = if tx_parts.len() == 2 {
(tx_parts[0], tx_parts[1])
} else {
(tx, tx)
};
let _ = self.set(key, value).unwrap();
ResponseDeliverTx {
code: 0,
data: Default::default(),
log: "".to_string(),
info: "".to_string(),
gas_wanted: 0,
gas_used: 0,
events: vec![Event {
fn commit(&self) -> ResponseCommit {
let (result_tx, result_rx) = channel();
channel_send(&self.cmd_tx, Command::Commit { result_tx }).unwrap();
let height = channel_recv(&result_rx).unwrap();
info!("Committed height {}", height);
ResponseCommit {
retain_height: height - 1,
}
}

fn finalize_block(&self, request: RequestFinalizeBlock) -> ResponseFinalizeBlock {
let mut events = Vec::new();
for tx in request.txs {
let tx = std::str::from_utf8(&tx).unwrap();
let tx_parts = tx.split('=').collect::<Vec<&str>>();
let (key, value) = if tx_parts.len() == 2 {
(tx_parts[0], tx_parts[1])
} else {
(tx, tx)
};
let _ = self.set(key, value).unwrap();
events.push(Event {
r#type: "app".to_string(),
attributes: vec![
EventAttribute {
Expand All @@ -214,19 +220,11 @@ impl Application for KeyValueStoreApp {
index: false,
},
],
}],
codespace: "".to_string(),
});
}
}

fn commit(&self) -> ResponseCommit {
let (result_tx, result_rx) = channel();
channel_send(&self.cmd_tx, Command::Commit { result_tx }).unwrap();
let (height, app_hash) = channel_recv(&result_rx).unwrap();
info!("Committed height {}", height);
ResponseCommit {
data: app_hash.into(),
retain_height: height - 1,
ResponseFinalizeBlock {
events,
..Default::default()
}
}
}
Expand Down Expand Up @@ -278,14 +276,14 @@ impl KeyValueStoreDriver {
}
}

fn commit(&mut self, result_tx: Sender<(i64, Vec<u8>)>) -> Result<(), Error> {
fn commit(&mut self, result_tx: Sender<i64>) -> Result<(), Error> {
// As in the Go-based key/value store, simply encode the number of
// items as the "app hash"
let mut app_hash = BytesMut::with_capacity(MAX_VARINT_LENGTH);
prost::encoding::encode_varint(self.store.len() as u64, &mut app_hash);
self.app_hash = app_hash.to_vec();
self.height += 1;
channel_send(&result_tx, (self.height, self.app_hash.clone()))
channel_send(&result_tx, self.height)
}
}

Expand All @@ -305,8 +303,8 @@ enum Command {
result_tx: Sender<Option<String>>,
},
/// Commit the current state of the application, which involves recomputing
/// the application's hash.
Commit { result_tx: Sender<(i64, Vec<u8>)> },
/// the application's hash, and return the new height.
Commit { result_tx: Sender<i64> },
}

fn channel_send<T>(tx: &Sender<T>, value: T) -> Result<(), Error> {
Expand Down
47 changes: 25 additions & 22 deletions abci/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

use std::net::{TcpStream, ToSocketAddrs};

use tendermint_proto::v0_37::abci::{
request, response, Request, RequestApplySnapshotChunk, RequestBeginBlock, RequestCheckTx,
RequestCommit, RequestDeliverTx, RequestEcho, RequestEndBlock, RequestFlush, RequestInfo,
use tendermint_proto::v0_38::abci::{
request, response, Request, RequestApplySnapshotChunk, RequestCheckTx, RequestCommit,
RequestEcho, RequestExtendVote, RequestFinalizeBlock, RequestFlush, RequestInfo,
RequestInitChain, RequestListSnapshots, RequestLoadSnapshotChunk, RequestOfferSnapshot,
RequestQuery, ResponseApplySnapshotChunk, ResponseBeginBlock, ResponseCheckTx, ResponseCommit,
ResponseDeliverTx, ResponseEcho, ResponseEndBlock, ResponseFlush, ResponseInfo,
ResponseInitChain, ResponseListSnapshots, ResponseLoadSnapshotChunk, ResponseOfferSnapshot,
ResponseQuery,
RequestQuery, RequestVerifyVoteExtension, ResponseApplySnapshotChunk, ResponseCheckTx,
ResponseCommit, ResponseEcho, ResponseExtendVote, ResponseFinalizeBlock, ResponseFlush,
ResponseInfo, ResponseInitChain, ResponseListSnapshots, ResponseLoadSnapshotChunk,
ResponseOfferSnapshot, ResponseQuery, ResponseVerifyVoteExtension,
};

use crate::{codec::ClientCodec, Error};
Expand Down Expand Up @@ -89,21 +89,6 @@ impl Client {
perform!(self, CheckTx, req)
}

/// Signal the beginning of a new block, prior to any `DeliverTx` calls.
pub fn begin_block(&mut self, req: RequestBeginBlock) -> Result<ResponseBeginBlock, Error> {
perform!(self, BeginBlock, req)
}

/// Apply a transaction to the application's state.
pub fn deliver_tx(&mut self, req: RequestDeliverTx) -> Result<ResponseDeliverTx, Error> {
perform!(self, DeliverTx, req)
}

/// Signal the end of a block.
pub fn end_block(&mut self, req: RequestEndBlock) -> Result<ResponseEndBlock, Error> {
perform!(self, EndBlock, req)
}

pub fn flush(&mut self) -> Result<ResponseFlush, Error> {
perform!(self, Flush, RequestFlush {})
}
Expand Down Expand Up @@ -142,6 +127,24 @@ impl Client {
perform!(self, ApplySnapshotChunk, req)
}

pub fn extend_vote(&mut self, req: RequestExtendVote) -> Result<ResponseExtendVote, Error> {
perform!(self, ExtendVote, req)
}

pub fn verify_vote_extension(
&mut self,
req: RequestVerifyVoteExtension,
) -> Result<ResponseVerifyVoteExtension, Error> {
perform!(self, VerifyVoteExtension, req)
}

pub fn finalize_block(
&mut self,
req: RequestFinalizeBlock,
) -> Result<ResponseFinalizeBlock, Error> {
perform!(self, FinalizeBlock, req)
}

fn perform(&mut self, req: request::Value) -> Result<response::Value, Error> {
self.codec.send(Request { value: Some(req) })?;
let res = self
Expand Down
2 changes: 1 addition & 1 deletion abci/src/codec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use std::{

use bytes::{Buf, BufMut, BytesMut};
use prost::Message;
use tendermint_proto::v0_37::abci::{Request, Response};
use tendermint_proto::v0_38::abci::{Request, Response};

use crate::error::Error;

Expand Down
2 changes: 1 addition & 1 deletion abci/src/error.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! tendermint-abci errors

use flex_error::{define_error, DisplayError};
use tendermint_proto::v0_37::abci::response::Value;
use tendermint_proto::v0_38::abci::response::Value;

define_error! {
Error {
Expand Down
2 changes: 1 addition & 1 deletion abci/tests/echo_app.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#[cfg(all(feature = "client", feature = "echo-app"))]
mod echo_app_integration {
use tendermint_abci::{ClientBuilder, EchoApp, ServerBuilder};
use tendermint_proto::v0_37::abci::RequestEcho;
use tendermint_proto::v0_38::abci::RequestEcho;

#[test]
fn echo() {
Expand Down
Loading