Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Make BEEFY client able to generate either ECDSA or (ECDSA,BLS) Finality proof. #13311

Closed
Closed
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
18c7d88
adding boilerplate for adding bls crypto
drskalman Dec 9, 2021
b86768c
First scratches to add bls crypto to primities/core and beefy.
drskalman Dec 10, 2021
fd75c93
suggested solution during BEEFY meeting for making BEEFY BLS compatible
drskalman Feb 15, 2022
322ad00
Merge remote-tracking branch 'origin/master' into skalman-bls-beefy
drskalman Feb 15, 2022
f7fca6d
- fix dependencies on arkwork and bls library.
drskalman Mar 10, 2022
5e8f2ea
to discuss HKDK with Al
drskalman May 9, 2022
2912fa1
resolve compilation errors for primitives/core/src/bls.rs
drskalman May 26, 2022
86b1e3a
resolve bls test compilation error, 6 tests still failing
drskalman May 31, 2022
2b59fa0
- change dependancy patches to repos available on github.
drskalman Jun 1, 2022
b602e7a
Merge branch 'master' of https://github.com/paritytech/substrate into…
drskalman Jun 1, 2022
68fd3a2
overwrite Cargo.lock from upstream and cargo-update to reslove Cargo.…
drskalman Jun 1, 2022
59d665e
add `bls_generate_new` and `bls_public_keys` to CryptoStore
drskalman Jun 14, 2022
857f85e
- implement host api `bls_public_keys, bls_generate, bls_sign,
drskalman Jun 14, 2022
a1ed112
- Adapt `primitives/beefy/src/commitment.rs and witness.rs` to have
drskalman Jun 28, 2022
b936150
implement TestNOPAggregatableSignature for Merkle based commitment.
drskalman Jul 4, 2022
1df925c
adapt an pass test for merkle root aggregator
drskalman Jul 5, 2022
28596f2
rust-fmtize
drskalman Jul 5, 2022
577cc68
add test for commitment containing aggregated bls signature.
drskalman Jul 7, 2022
0f53f28
make different traits for keystore for ecdsa-only and ecdsa-bls beefy…
drskalman Jul 18, 2022
fed0bbc
reverse the idea of validator aggregating received bls signature
drskalman Jul 21, 2022
7de89ae
make beefy primitve test compile and pass
drskalman Jul 23, 2022
5379e62
implement `should_convert_dually_signed_commitment_to_witness` test
drskalman Jul 25, 2022
7bdb082
Merge branch 'skalman-bls-beefy' into skalman-bls-beefy-client
drskalman Jul 25, 2022
1661c19
separate impl for ECDSA and BLS keystore
drskalman Aug 5, 2022
6a1c26f
Merge branch 'skalman-bls-beefy-client' of https://github.com/drskalm…
drskalman Aug 11, 2022
ab682ce
add bls support to LocalKeyStore
drskalman Aug 16, 2022
231a38e
implement `BeefyBLSKeystore` and `BeefyBLSnECDSAKeystore`
drskalman Aug 16, 2022
1847516
Make BeefyKeyStore generic for Beefy Worker
drskalman Aug 24, 2022
cc3a9d6
add BeefyKeystore generic type to GossipValidator so it can turn
drskalman Aug 30, 2022
e0d8740
adapt BEEFY GossipValidator to a generic Keystore
drskalman Sep 1, 2022
ea40ab9
- resolve errors in ECDSAnBLSKeystore implementation.
drskalman Sep 9, 2022
9349cac
clone keys on the fly while iterating.
drskalman Sep 12, 2022
787f80d
- rename `beefy::crypto` to `beefy:ecdsa_crypto`.
drskalman Sep 16, 2022
ab1c2f3
Merge branch 'skalman-bls-beefy' into skalman-bls-beefy-client
drskalman Sep 16, 2022
f48c8eb
- Make `BeefyApi` trait to distinguish between AuthorityId and PublicKey
drskalman Sep 19, 2022
d7044f8
share changes to beefy client to find solution to authority id problem
drskalman Sep 21, 2022
524debf
adapt Beefy Worker, GossipValidator and Round to Generic AuthId and T…
drskalman Sep 22, 2022
c8db18f
- Resolve conflict between Public and AuthId
drskalman Sep 24, 2022
95f6845
Merge branch 'master' into skalman-beefy-authority-id-adventures
drskalman Sep 26, 2022
50b2f9b
- updated tests to new bls library changes.
drskalman Oct 17, 2022
4c905a8
Merge branch 'master' into skalman-beefy-authority-id-adventures
drskalman Oct 18, 2022
5c89709
make all new Network and Communication objects depends on `AuthId` a…
drskalman Oct 18, 2022
782734d
- do not depend on rand default features (std).
drskalman Oct 26, 2022
2b9cc3c
minor changes and remove patching parity common.
drskalman Oct 27, 2022
8ebb239
move bls-like branch specification from patch to dep due to the cargo
drskalman Oct 29, 2022
a147238
- make all beefy tests generic over AuthityId, Signature and Keystore.
drskalman Oct 31, 2022
6cdc680
make `create_worker` generic on Keystore
drskalman Nov 4, 2022
95e283a
make Keyring generic over key type - down to 97 errors
drskalman Nov 25, 2022
d93fca0
implement keyring enum instead of wrapping it in a struct
drskalman Dec 8, 2022
accfaf7
- Re-apply BeefyVerify changes.
drskalman Dec 8, 2022
61d5168
First step in implementing Pair trait for ECDSAnBLS crypto
drskalman Dec 11, 2022
b03a4ce
- introduce `SimpleKeyPair` to make `Keyring` generic over key pair type
drskalman Dec 16, 2022
015e2a9
fix over hashing for bls_sign in the keystore
drskalman Dec 16, 2022
5022b7e
beefy `public_keys_works` test fails because the keystore forgets the…
drskalman Dec 19, 2022
2add57c
make testing infrastructure in `client/beefy/src/tests.rs` generic ov…
drskalman Jan 11, 2023
670c0cf
implement `beefy_finalizing_blocks_using_ecdsa_signature` though it o…
drskalman Jan 16, 2023
c6acac6
- implement beefy_finalizing_blocks_using_ecdsa_n_bls_signature
drskalman Jan 17, 2023
789a5a8
- making `new_finality_proof`,`lagging_validators`,`correct_beefy_pay…
drskalman Jan 17, 2023
1165d7c
make `voter_initialization`, `on_demand_beefy_justification_sync` gen…
drskalman Jan 17, 2023
1aaf6be
- Re-active tests and make helpers in beefy/worker.rs to crypto
drskalman Jan 18, 2023
59f10a4
implement keystore_vs_validator_set test for ecdsa and ecdsa_n_bls pa…
drskalman Jan 18, 2023
84a9902
depends on arkworks 0.4.0
drskalman Jan 25, 2023
744fe42
- use `store.authority_id` istead of key.contains in `keystore::publi…
drskalman Jan 29, 2023
6eaa95d
make beefy justification tests generic over keytype
drskalman Jan 30, 2023
18d5a59
- make rounds tests generic over key type
drskalman Jan 30, 2023
a010852
Merge branch 'master' into skalman-beefy-authority-id-adventures
drskalman Jan 31, 2023
d323ca7
make beefy client build after merging with master.
drskalman Feb 2, 2023
e53c9c8
adapt old test to new tokio style. new tests has been commented out.
drskalman Feb 3, 2023
d0c790a
new beefy tests in worker and tests adapted to generic key model
drskalman Feb 3, 2023
e08387f
- specify `AuthId` and `TSignature` when calling `load_persitent` oth…
drskalman Feb 4, 2023
3be0a82
get rid of redundant imports and fix all warnings.
drskalman Feb 4, 2023
42c894b
cargo fmt beefy primitives
drskalman Feb 4, 2023
469aadb
cargo fmt beefy client, keystore client and primitives, primitives
drskalman Feb 4, 2023
73657c5
Add `AuthoritySetCommitment` field to `BeefyAuthoritySet` needed by A…
drskalman Feb 24, 2023
096f94d
- depend on default branch of bls-like and apk-proofs.
drskalman Mar 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,082 changes: 685 additions & 397 deletions Cargo.lock

Large diffs are not rendered by default.

12 changes: 12 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -321,3 +321,15 @@ inherits = "release"
lto = "fat"
# https://doc.rust-lang.org/rustc/codegen-options/index.html#codegen-units
codegen-units = 1

[patch.crates-io]
bls-like = {git = "https://github.com/w3f/bls", branch = "skalman-hash-to-curve-wb"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This doesn't seem to be needed since everywhere it's used like bls-like = {git = "https://github.com/w3f/bls", branch = "skalman-hash-to-curve-wb"}. However it would make sense to use it like bls-like = {git = "https://github.com/w3f/bls", branch = "master"} in other places in which case, this patch would be needed.

Also is this library reviewed and production-ready ? Or should we also review it ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably deprecate that whole crate myself. It's a legacy design that pre-dates our gossip optimizations in https://eprint.iacr.org/2022/1611 and the apk proofs work.

As syed do the hash-to-curve work, it should work fine, and comply with the proposed standard such as it is, but it makes verifying BLS signatures quite slow (like everyone else's BLS.. We know how to do this faster.

# ark-ec = { git = "https://github.com/w3f/arkworks-algebra", branch = 'master'}
# ark-ff = { git = "https://github.com/w3f/arkworks-algebra", branch = 'master'}
# ark-std = { git = "https://github.com/arkworks-rs/std"}
# ark-serialize = { git = "https://github.com/w3f/arkworks-algebra", branch = 'master'}
# ark-serialize-derive = { git = "https://github.com/w3f/arkworks-algebra", branch = 'master'}
# ark-bls12-381 = { git = "https://github.com/w3f/arkworks-curves", branch = 'skalman-bls12-377-wb-hash2curve'}
# ark-bls12-377 = { git = "https://github.com/w3f/arkworks-curves", branch = 'skalman-bls12-377-wb-hash2curve'}

# primitive-types = { git = "https://github.com/drskalman/parity-common"}
1 change: 1 addition & 0 deletions client/beefy/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,4 @@ sp-finality-grandpa = { version = "4.0.0-dev", path = "../../primitives/finality
sp-keyring = { version = "7.0.0", path = "../../primitives/keyring" }
sp-tracing = { version = "6.0.0", path = "../../primitives/tracing" }
substrate-test-runtime-client = { version = "2.0.0", path = "../../test-utils/runtime/client" }
smart-default = "0.6.0"
72 changes: 61 additions & 11 deletions client/beefy/src/aux_schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ use log::{info, trace};
use sc_client_api::{backend::AuxStore, Backend};
use sp_blockchain::{Error as ClientError, Result as ClientResult};
use sp_runtime::traits::Block as BlockT;
use std::fmt::Debug;

const VERSION_KEY: &[u8] = b"beefy_auxschema_version";
const WORKER_STATE: &[u8] = b"beefy_voter_state";
Expand All @@ -36,15 +37,20 @@ pub(crate) fn write_current_version<B: AuxStore>(backend: &B) -> ClientResult<()
}

/// Write voter state.
pub(crate) fn write_voter_state<Block: BlockT, B: AuxStore>(
pub(crate) fn write_voter_state<
Block: BlockT,
B: AuxStore,
AuthId: Encode + Decode + Debug + Clone + Ord + Sync + Send + std::hash::Hash,
TSignature: Encode + Decode + Debug + Clone + Sync + Send,
>(
backend: &B,
state: &PersistedState<Block>,
state: &PersistedState<Block, AuthId, TSignature>,
) -> ClientResult<()> {
trace!(target: LOG_TARGET, "🥩 persisting {:?}", state);
backend.insert_aux(&[(WORKER_STATE, state.encode().as_slice())], &[])
}

fn load_decode<B: AuxStore, T: Decode>(backend: &B, key: &[u8]) -> ClientResult<Option<T>> {
fn load_decode<BE: AuxStore, T: Decode>(backend: &BE, key: &[u8]) -> ClientResult<Option<T>> {
match backend.get_aux(key)? {
None => Ok(None),
Some(t) => T::decode(&mut &t[..])
Expand All @@ -54,16 +60,21 @@ fn load_decode<B: AuxStore, T: Decode>(backend: &B, key: &[u8]) -> ClientResult<
}

/// Load or initialize persistent data from backend.
pub(crate) fn load_persistent<B, BE>(backend: &BE) -> ClientResult<Option<PersistedState<B>>>
pub(crate) fn load_persistent<B, BE, AuthId, TSignature>(
backend: &BE,
) -> ClientResult<Option<PersistedState<B, AuthId, TSignature>>>
where
B: BlockT,
BE: Backend<B>,
AuthId: Encode + Decode + Debug + Clone + Ord + Sync + Send + std::hash::Hash,
TSignature: Encode + Decode + Debug + Clone + Sync + Send,
{
let version: Option<u32> = load_decode(backend, VERSION_KEY)?;

match version {
None => (),
Some(1) => return load_decode::<_, PersistedState<B>>(backend, WORKER_STATE),
Some(1) =>
return load_decode::<_, PersistedState<B, AuthId, TSignature>>(backend, WORKER_STATE),
other =>
return Err(ClientError::Backend(format!("Unsupported BEEFY DB version: {:?}", other))),
}
Expand All @@ -75,31 +86,70 @@ where
#[cfg(test)]
pub(crate) mod tests {
use super::*;
use crate::tests::BeefyTestNet;
use crate::tests::{BeefyAuthIdMaker, BeefyTestNet};
use sc_network_test::TestNetFactory;

use crate::keystore::{BeefyBLSnECDSAKeystore, BeefyECDSAKeystore, BeefyKeystore};

use beefy_primitives::{
bls_crypto::{Public as BLSPublic, Signature as BLSSignature},
ecdsa_crypto::{Public as ECDSAPublic, Signature as ECDSASignature},
};

// also used in tests.rs
pub fn verify_persisted_version<B: BlockT, BE: Backend<B>>(backend: &BE) -> bool {
let version: u32 = load_decode(backend, VERSION_KEY).unwrap().unwrap();
version == CURRENT_VERSION
}

#[tokio::test]
async fn should_load_persistent_sanity_checks() {
let mut net = BeefyTestNet::new(1);
//async fn should_load_persistent_sanity_checks()
async fn should_load_persistent_sanity_checks<AuthId, TSignature, TBeefyKeystore>()
where
TBeefyKeystore: BeefyKeystore<AuthId, TSignature, Public = AuthId> + 'static,
AuthId: Clone
+ Encode
+ Decode
+ Debug
+ Ord
+ Sync
+ Send
+ BeefyAuthIdMaker
+ std::hash::Hash
+ 'static,
TSignature: Encode + Decode + Debug + Clone + Sync + Send + std::cmp::PartialEq + 'static,
{
let mut net: BeefyTestNet<ECDSAPublic, ECDSASignature, BeefyECDSAKeystore> =
BeefyTestNet::new(1);
//let mut net: BeefyTestNet<AuthId, TSignature, TBeefyKeystore> = BeefyTestNet::new(1);
let backend = net.peer(0).client().as_backend();

// version not available in db -> None
assert_eq!(load_persistent(&*backend).unwrap(), None);
assert_eq!(load_persistent::<_, _, AuthId, TSignature>(&*backend).unwrap(), None);

// populate version in db
write_current_version(&*backend).unwrap();
// verify correct version is retrieved
assert_eq!(load_decode(&*backend, VERSION_KEY).unwrap(), Some(CURRENT_VERSION));

// version is available in db but state isn't -> None
assert_eq!(load_persistent(&*backend).unwrap(), None);
assert_eq!(load_persistent::<_, _, AuthId, TSignature>(&*backend).unwrap(), None);

// full `PersistedState` load is tested in `tests.rs`.
}

#[tokio::test]
async fn should_load_persistent_sanity_checks_with_ecdsa_signature() {
should_load_persistent_sanity_checks::<ECDSAPublic, ECDSASignature, BeefyECDSAKeystore>()
.await;
}

#[tokio::test]
async fn should_load_persistent_sanity_checks_with_ecdsa_n_bls_signature() {
should_load_persistent_sanity_checks::<
(ECDSAPublic, BLSPublic),
(ECDSASignature, BLSSignature),
BeefyBLSnECDSAKeystore,
>()
.await;
}
}
Loading