Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gossipsub message signing #1583

Merged
merged 43 commits into from
Aug 3, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
aeba523
Gossipsub message signing
AgeManning May 19, 2020
573e4b6
Update ipfs-private example
AgeManning May 19, 2020
d2babd4
Add optimisation to prevent key calculation each message
AgeManning May 22, 2020
ac3c977
Handle forwarding of signed messages and reviewers comments
AgeManning May 25, 2020
e9afd95
Merge branch 'master' into gossipsub-signing
AgeManning May 27, 2020
3d1a025
protocols/gossipsub/src/protocol: Use quickcheck to test signing (#31)
mxinden Jun 1, 2020
31b6849
Shift signing into behaviour
AgeManning Jun 1, 2020
a8cec0b
Merge latest master
AgeManning Jul 1, 2020
2b38112
Address reviewers suggestions
AgeManning Jul 1, 2020
7d678f5
Send subscriptions to all peers
AgeManning Jul 1, 2020
dc050dd
Update examples
AgeManning Jul 1, 2020
5bf1cfc
Shift signing option into the behaviour
AgeManning Jul 1, 2020
4ffe728
Revert changes to core/
AgeManning Jul 9, 2020
24b738c
Reviewers suggestion
AgeManning Jul 9, 2020
a2e4bf0
Merge remote-tracking branch 'origin/master' into gossipsub-signing
AgeManning Jul 9, 2020
2f20571
Merge latest master
AgeManning Jul 23, 2020
2c50ba2
Revert changes to core/
AgeManning Jul 23, 2020
4de878e
Switch gossipsub state to use sets instead of vectors
rklaehn Jul 23, 2020
e43f0a0
Make tests pass
rklaehn Jul 23, 2020
57f07aa
Some clippy
rklaehn Jul 23, 2020
b6eeb3c
Prevent duplicate finding during JOIN
AgeManning Jul 24, 2020
0d5bde4
Merge #1675
AgeManning Jul 24, 2020
aec51e0
Update tests, improve logging
AgeManning Jul 24, 2020
759a5ad
Update the default d_lo constant
AgeManning Jul 24, 2020
495ffe4
Handle errors correctly
AgeManning Jul 24, 2020
1ea2dd9
Add privacy and validation options in the config
AgeManning Jul 24, 2020
1f73ded
A few improvements (#32)
rklaehn Jul 24, 2020
7b76c8f
Improve signing validation logic
AgeManning Jul 24, 2020
f5d32b2
Change the gossipsub rpc protocol to use bytes for message ids (#34)
rklaehn Jul 25, 2020
c3a7757
Add optional privacy and validation settings
AgeManning Jul 26, 2020
0c43ed4
Correct doc link
AgeManning Jul 27, 2020
efa40e3
Prevent invalid messages from being gossiped
AgeManning Jul 27, 2020
a514c60
Remove unvalidated messages from gossip. Reintroduce duplicate cache
AgeManning Jul 27, 2020
b257025
Send grafts when subscriptions are added to the mesh
AgeManning Jul 28, 2020
b20771b
Merge latest master
AgeManning Jul 28, 2020
2f4a50a
Only add messages to memcache if not duplicates
AgeManning Jul 28, 2020
be7c9e2
Add mesh maintenance tests and remove excess peers from mesh
AgeManning Jul 29, 2020
e049ff7
Apply reviewers suggestions
AgeManning Jul 29, 2020
74d2779
Merge latest master
AgeManning Jul 29, 2020
6bac865
Wrap comments
AgeManning Jul 29, 2020
94469cf
Ensure sequence number is sent
AgeManning Aug 2, 2020
60f8a1e
Merge latest master
AgeManning Aug 2, 2020
fef66dd
Maintain the debug trait
AgeManning Aug 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 93 additions & 20 deletions protocols/gossipsub/src/behaviour.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,17 @@ use crate::handler::GossipsubHandler;
use crate::mcache::MessageCache;
use crate::protocol::{
GossipsubControlAction, GossipsubMessage, GossipsubSubscription, GossipsubSubscriptionAction,
MessageId,
MessageId, SIGNING_PREFIX,
};
use crate::rpc_proto;
use crate::topic::{Topic, TopicHash};
use futures::prelude::*;
use libp2p_core::{connection::ConnectionId, identity::Keypair, Multiaddr, PeerId};
use libp2p_swarm::{
NetworkBehaviour, NetworkBehaviourAction, NotifyHandler, PollParameters, ProtocolsHandler,
};
use log::{debug, error, info, trace, warn};
use prost::Message;
use rand;
use rand::{seq::SliceRandom, thread_rng};
use std::{
Expand Down Expand Up @@ -84,23 +86,51 @@ pub struct Gossipsub {

/// Heartbeat interval stream.
heartbeat: Interval,

/// The peer_id public to add to messages if signing is enabled and the current libp2p key is
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
/// not inlined in the `PeerId`.
key: Option<Vec<u8>>,
}

impl Gossipsub {
/// Creates a `Gossipsub` struct given a set of parameters specified by `gs_config`.
pub fn new(keypair: Keypair, gs_config: GossipsubConfig) -> Self {
// Set up the router given the configuration settings.

// Sets up the source_id of published messages.
let message_source_id = if gs_config.no_source_id {
PeerId::from_bytes(crate::config::IDENTITY_SOURCE.to_vec()).expect("Valid peer id")
mxinden marked this conversation as resolved.
Show resolved Hide resolved
} else {
keypair.public().into_peer_id()
};

// If signing is not enabled, we don't need to store the keypair.
let keypair = if gs_config.disable_message_signing {
None
} else {
Some(keypair)
};

// If we are signing and the public key cannot be inlined in the `PeerId`, we store the
// public key to attach to published messages.
let key = {
if let Some(keypair) = keypair.as_ref() {
// signing is required
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
let key_enc = keypair.public().into_protobuf_encoding();
if key_enc.len() <= 42 {
// The public key can be inlined in [`rpc_proto::Message::from`], so we don't include it
// specifically in the [`rpc_proto::Message::key`] field.
None
} else {
// Include the protobuf encoding of the public key in the message
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
Some(key_enc)
}
} else {
// Signing is not required, no key needs to be added
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
None
}
};

Gossipsub {
config: gs_config.clone(),
events: VecDeque::new(),
Expand All @@ -121,6 +151,7 @@ impl Gossipsub {
Instant::now() + gs_config.heartbeat_initial_delay,
gs_config.heartbeat_interval,
),
key,
}
}

Expand Down Expand Up @@ -225,18 +256,18 @@ impl Gossipsub {
/// Publishes a message with multiple topics to the network.
pub fn publish_many(
&mut self,
topic: impl IntoIterator<Item = Topic>,
topics: impl IntoIterator<Item = Topic>,
data: impl Into<Vec<u8>>,
) {
let message = GossipsubMessage {
source: self.message_source_id.clone(),
data: data.into(),
// To be interoperable with the go-implementation this is treated as a 64-bit
// big-endian uint.
sequence_number: rand::random(),
topics: topic.into_iter().map(|t| self.topic_hash(t)).collect(),
signature: None, // signature will get created when being published
key: None,
let message = match self.build_message(
topics.into_iter().map(|t| self.topic_hash(t)).collect(),
data.into(),
) {
Ok(m) => m,
Err(e) => {
error!("{}", e);
return;
}
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
};

let msg_id = (self.config.message_id_fn)(&message);
Expand All @@ -251,10 +282,7 @@ impl Gossipsub {
return;
}

debug!(
"Publishing message: {:?}",
(self.config.message_id_fn)(&message)
);
debug!("Publishing message: {:?}", msg_id);

// Forward the message to mesh peers
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
let message_source = &self.message_source_id.clone();
Expand Down Expand Up @@ -701,8 +729,8 @@ impl Gossipsub {
// too little peers - add some
if peers.len() < self.config.mesh_n_low {
debug!(
"HEARTBEAT: Mesh low. Topic: {:?} Contains: {:?} needs: {:?}",
topic_hash.clone().into_string(),
"HEARTBEAT: Mesh low. Topic: {} Contains: {} needs: {}",
topic_hash,
peers.len(),
self.config.mesh_n_low
);
Expand All @@ -724,7 +752,7 @@ impl Gossipsub {
// too many peers - remove some
if peers.len() > self.config.mesh_n_high {
debug!(
"HEARTBEAT: Mesh high. Topic: {:?} Contains: {:?} needs: {:?}",
"HEARTBEAT: Mesh high. Topic: {} Contains: {} needs: {}",
topic_hash,
peers.len(),
self.config.mesh_n_high
Expand Down Expand Up @@ -946,6 +974,52 @@ impl Gossipsub {
debug!("Completed forwarding message");
}

/// Constructs a `GossipsubMessage` performing message signing if required.
pub(crate) fn build_message(
&self,
topics: Vec<TopicHash>,
data: Vec<u8>,
) -> Result<GossipsubMessage, String> {
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
let sequence_number: u64 = rand::random();
let message = rpc_proto::Message {
from: Some(self.message_source_id.clone().into_bytes()),
data: Some(data),
seqno: Some(sequence_number.to_be_bytes().to_vec()),
topic_ids: topics.clone().into_iter().map(|t| t.into()).collect(),
signature: None,
key: None,
};

// If a signature is required, generate it
let signature = if let Some(keypair) = self.keypair.as_ref() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

message is only needed when Some(keypair) == self.keypair.as_ref() right?

Suggested change
let message = rpc_proto::Message {
from: Some(self.message_source_id.clone().into_bytes()),
data: Some(data),
seqno: Some(sequence_number.to_be_bytes().to_vec()),
topic_ids: topics.clone().into_iter().map(|t| t.into()).collect(),
signature: None,
key: None,
};
// If a signature is required, generate it
let signature = if let Some(keypair) = self.keypair.as_ref() {
// If a signature is required, generate it
let signature = if let Some(keypair) = self.keypair.as_ref() {
let message = rpc_proto::Message {
from: Some(self.message_source_id.clone().into_bytes()),
data: Some(data),
seqno: Some(sequence_number.to_be_bytes().to_vec()),
topic_ids: topics.clone().into_iter().map(|t| t.into()).collect(),
signature: None,
key: None,
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is true, but if i recall correctly, I was trying to avoid the clone on data.
I'll change it, but it now clones the data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I guess as this is only scoped to the function itself we can delay the decision. In case it shows up in benchmarks we can easily change it. Rustc or LLVM might as well just optimize it away, not sure.

let mut buf = Vec::with_capacity(message.encoded_len());
message
.encode(&mut buf)
.expect("Buffer has sufficient capacity");
// the signature is over the bytes "libp2p-pubsub:<protobuf-message>"
let mut signature_bytes = SIGNING_PREFIX.to_vec();
signature_bytes.extend_from_slice(&buf);
Some(
keypair
.sign(&signature_bytes)
.map_err(|e| format!("Signing error: {}", e))?,
)
} else {
None
};

Ok(GossipsubMessage {
source: self.message_source_id.clone(),
data: message.data.expect("data exists"),
// To be interoperable with the go-implementation this is treated as a 64-bit
// big-endian uint.
sequence_number,
topics,
signature,
key: self.key.clone(),
})
}

/// Helper function to get a set of `n` random gossipsub peers for a `topic_hash`
/// filtered by the function `f`.
fn get_random_peers(
Expand Down Expand Up @@ -1020,9 +1094,8 @@ impl NetworkBehaviour for Gossipsub {
fn new_handler(&mut self) -> Self::ProtocolsHandler {
GossipsubHandler::new(
self.config.protocol_id.clone(),
self.message_source_id.clone(),
self.config.max_transmit_size,
self.keypair.clone(),
self.keypair.is_some(), // if a keypair is stored we want to verify signatures
)
}

Expand Down
8 changes: 2 additions & 6 deletions protocols/gossipsub/src/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ use crate::behaviour::GossipsubRpc;
use crate::protocol::{GossipsubCodec, ProtocolConfig};
use futures::prelude::*;
use futures_codec::Framed;
use libp2p_core::identity::Keypair;
use libp2p_core::upgrade::{InboundUpgrade, OutboundUpgrade};
use libp2p_core::PeerId;
use libp2p_swarm::protocols_handler::{
KeepAlive, ProtocolsHandler, ProtocolsHandlerEvent, ProtocolsHandlerUpgrErr, SubstreamProtocol,
};
Expand Down Expand Up @@ -84,16 +82,14 @@ impl GossipsubHandler {
/// Builds a new `GossipsubHandler`.
pub fn new(
protocol_id: impl Into<Cow<'static, [u8]>>,
local_peer_id: PeerId,
max_transmit_size: usize,
keypair: Option<Keypair>,
verify_signatures: bool,
) -> Self {
GossipsubHandler {
listen_protocol: SubstreamProtocol::new(ProtocolConfig::new(
protocol_id,
local_peer_id,
max_transmit_size,
keypair,
verify_signatures,
)),
inbound_substream: None,
outbound_substream: None,
Expand Down
Loading