Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(perf): implement libp2p perf protocol #3508

Merged
merged 56 commits into from
Mar 19, 2023
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
43a4f3e
Scaffolding
mxinden Feb 25, 2023
d843caa
Scaffolding for client protocol
mxinden Feb 25, 2023
261922d
Implement client send receive
mxinden Feb 25, 2023
26e26d3
Scaffolding for end-to-end test
mxinden Feb 25, 2023
a31e69a
Implement server side protocol
mxinden Feb 25, 2023
04858c6
Bubble events up to user
mxinden Feb 25, 2023
e1a0ced
Track stats for single run
mxinden Feb 25, 2023
a41f41b
Add client and server example
mxinden Feb 26, 2023
a0a55ae
Track time by write and read
mxinden Feb 26, 2023
1c3bfe9
Don't allocate in protocol implementation
mxinden Feb 26, 2023
b95f8a6
Don't panic on stream negotiation failure
mxinden Feb 26, 2023
9bbd99a
Add support for QUIC
mxinden Feb 26, 2023
761df78
Fix crate description
mxinden Mar 3, 2023
3b68b37
Move examples into src/bin/
mxinden Mar 3, 2023
ffdf089
Handle inbound connection in client behaviour
mxinden Mar 3, 2023
39694af
Don't do connection management in perf behaviour
mxinden Mar 3, 2023
4cab07d
Adjust tests and don't depend on DummyHandler
mxinden Mar 3, 2023
b9ef80f
Implement keep alive handling
mxinden Mar 3, 2023
489fbfe
Have server emit event with run stats
mxinden Mar 3, 2023
96e7997
Handle outbound error on client handler
mxinden Mar 4, 2023
27586dc
Handle listen error in client handler
mxinden Mar 4, 2023
3fbd70e
Handle stream errors on server handler
mxinden Mar 4, 2023
8b52690
Handle FromSwarm events in client behaviour
mxinden Mar 4, 2023
d9ee033
Adjust tests
mxinden Mar 4, 2023
8c43498
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf
mxinden Mar 4, 2023
ded45c5
Fix doc comment
mxinden Mar 4, 2023
d91a4f8
Fix clippy warnings
mxinden Mar 4, 2023
e023e89
Update rust version
mxinden Mar 4, 2023
5cd617d
Revert .dockerignore and use dedicated run cache
mxinden Mar 7, 2023
2efd0a9
Use async-std main instead of block_on
mxinden Mar 7, 2023
fbca42e
Don't print peer ID in debug
mxinden Mar 7, 2023
ce57655
Use Display for multiaddr
mxinden Mar 7, 2023
85bb764
Make submodules of server and client private
mxinden Mar 7, 2023
bb1b553
Use .. on ConnectionEstablished
mxinden Mar 7, 2023
ade98eb
Use instant::Instant
mxinden Mar 7, 2023
4e35494
Update protocols/perf/src/server/behaviour.rs
mxinden Mar 7, 2023
96327ba
Update protocols/perf/src/client/behaviour.rs
mxinden Mar 7, 2023
d7c9328
Make server Behaviour Event a struct thus remove Finished
mxinden Mar 7, 2023
74dd50e
Implement handle_established_outbound_connection or server
mxinden Mar 7, 2023
a05272e
Make Event of server handler a struct
mxinden Mar 7, 2023
3182903
Use Void in server handler
mxinden Mar 7, 2023
d0e96d2
Use Void for server handler error
mxinden Mar 7, 2023
cd8baf6
Use Void for client handler errror
mxinden Mar 7, 2023
8ef5de8
Don't panic but log error in server handler on stream io
mxinden Mar 7, 2023
82eb107
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf
mxinden Mar 7, 2023
b14f6b8
Fix clippy lint
mxinden Mar 7, 2023
5f3e382
Don't expose `Handler` publicly
thomaseizinger Mar 8, 2023
ba96227
Remove .dockerignore diff
mxinden Mar 13, 2023
64b6498
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf
mxinden Mar 13, 2023
9519562
Use libp2p-swarm-test
mxinden Mar 13, 2023
3b3b08f
Add newline to Dockerfile
mxinden Mar 13, 2023
ea30c1b
Use async_std::test and libp2p-swarm-test wait
mxinden Mar 19, 2023
a98ca38
Limit the number of decimals for floats in output
mxinden Mar 19, 2023
cd846b1
Log start and connection establishment
mxinden Mar 19, 2023
2651d72
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf
mxinden Mar 19, 2023
75065bd
Add changelog
mxinden Mar 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
target
Copy link
Contributor

@thomaseizinger thomaseizinger Mar 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this will send the entire target directory to docker which can be dozens of GB if you don't run cargo clean regularly.

We should use a dedicated run cache instead to speed up the build of docker files. See "Use a dedicated RUN cache": https://docs.docker.com/build/cache/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this was meant for debugging only. Thanks for highlighting it.

5cd617d reverts the change and uses the dedicated run caches. Mind taking another look @thomaseizinger?

23 changes: 23 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ full = [
"metrics",
"mplex",
"noise",
"perf",
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
"ping",
"plaintext",
"pnet",
Expand Down Expand Up @@ -64,6 +65,7 @@ mdns = ["dep:libp2p-mdns"]
metrics = ["dep:libp2p-metrics"]
mplex = ["dep:libp2p-mplex"]
noise = ["dep:libp2p-noise"]
perf = ["dep:libp2p-perf"]
ping = ["dep:libp2p-ping", "libp2p-metrics?/ping"]
plaintext = ["dep:libp2p-plaintext"]
pnet = ["dep:libp2p-pnet"]
Expand Down Expand Up @@ -101,6 +103,7 @@ libp2p-kad = { version = "0.43.0", path = "protocols/kad", optional = true }
libp2p-metrics = { version = "0.12.0", path = "misc/metrics", optional = true }
libp2p-mplex = { version = "0.39.0", path = "muxers/mplex", optional = true }
libp2p-noise = { version = "0.42.0", path = "transports/noise", optional = true }
libp2p-perf = { version = "0.1.0", path = "protocols/perf", optional = true }
libp2p-ping = { version = "0.42.0", path = "protocols/ping", optional = true }
libp2p-plaintext = { version = "0.39.0", path = "transports/plaintext", optional = true }
libp2p-pnet = { version = "0.22.2", path = "transports/pnet", optional = true }
Expand Down Expand Up @@ -159,6 +162,7 @@ members = [
"protocols/identify",
"protocols/kad",
"protocols/mdns",
"protocols/perf",
"protocols/ping",
"protocols/relay",
"protocols/request-response",
Expand Down
38 changes: 38 additions & 0 deletions protocols/perf/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[package]
name = "libp2p-perf"
edition = "2021"
rust-version = "1.64.0"
description = "libp2p perf protocol implementation"
version = "0.1.0"
authors = ["Max Inden <mail@max-inden.de>"]
license = "MIT"
repository = "https://github.com/libp2p/rust-libp2p"
keywords = ["peer-to-peer", "libp2p", "networking"]
categories = ["network-programming", "asynchronous"]

[dependencies]
anyhow = "1"
clap = { version = "4.1.6", features = ["derive"] }
env_logger = "0.10.0"
futures = "0.3.26"
libp2p-core = { version = "0.39.0", path = "../../core" }
libp2p-dns = { version = "0.39.0", path = "../../transports/dns", features = ["async-std"] }
libp2p-noise = { version = "0.42.0", path = "../../transports/noise" }
libp2p-quic = { version = "0.7.0-alpha.2", path = "../../transports/quic", features = ["async-std"] }
libp2p-swarm = { version = "0.42.0", path = "../../swarm", features = ["macros", "async-std"] }
libp2p-tcp = { version = "0.39.0", path = "../../transports/tcp", features = ["async-io"] }
libp2p-yamux = { version = "0.43.0", path = "../../muxers/yamux" }
Copy link
Contributor

@thomaseizinger thomaseizinger Mar 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need most these for the binaries right? How about we introduce a separate crate that implements the server and client binaries, similarly to interop-tests? That would allow this protocol to be WASM-friendly. Plus, in the spirit of #3111, it would be consistent with not introduce binaries in our protocol crates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which folder should these binaries go into?

That would allow this protocol to be WASM-friendly.

I don't think we should introduce any complexity for the sake of supporting WASM at this point. It is not a critical protocol. I don't see it used in WASM any time soon.

Plus, in the spirit of #3111, it would be consistent with not introduce binaries in our protocol crates.

Consistency is a good argument in my eyes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which folder should these binaries go into?

You'd have to make another crate, like interop-tests. Perhaps perf-utils, perf-cs (for perf-client-server) or perf-bins?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And perf-bins/ would be a top level folder? Is that worth the noise it is introducing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could put them into misc/ too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomaseizinger do you feel strongly about this? Thinking about it some more, I don't think the split into two folders is worth the effort at this point. I expect this crate to change quite a bit after this pull request merges.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

log = "0.4"
thiserror = "1.0"
void = "1"

[dev-dependencies]
rand = "0.8"
libp2p-plaintext = { path = "../../transports/plaintext" }

# Passing arguments to the docsrs builder in order to properly document cfg's.
# More information: https://docs.rs/about/builds#cross-compiling
[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]
rustc-args = ["--cfg", "docsrs"]
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 4 additions & 0 deletions protocols/perf/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM ubuntu:kinetic
ADD ./target/release/perf-client /usr/local/bin/perf-client
ADD ./target/release/perf-server /usr/local/bin/perf-server
ENTRYPOINT [ "perf-server"]
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
139 changes: 139 additions & 0 deletions protocols/perf/src/bin/perf-client.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
// Copyright 2023 Protocol Labs.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
// DEALINGS IN THE SOFTWARE.

use anyhow::{bail, Result};
use clap::Parser;
use futures::{executor::block_on, future::Either, StreamExt};
use libp2p_core::{
identity, muxing::StreamMuxerBox, transport::OrTransport, upgrade, Multiaddr, PeerId, Transport,
};
use libp2p_dns::DnsConfig;
use libp2p_perf::client::RunParams;
use libp2p_swarm::{SwarmBuilder, SwarmEvent};
use log::info;

#[derive(Debug, Parser)]
#[clap(name = "libp2p perf client")]
struct Opts {
#[arg(long)]
server_address: Multiaddr,
}

fn main() -> Result<()> {
env_logger::init();

let opts = Opts::parse();

// Create a random PeerId
let local_key = identity::Keypair::generate_ed25519();
let local_peer_id = PeerId::from(local_key.public());

let transport = {
let tcp =
libp2p_tcp::async_io::Transport::new(libp2p_tcp::Config::default().port_reuse(true))
.upgrade(upgrade::Version::V1Lazy)
.authenticate(
libp2p_noise::NoiseAuthenticated::xx(&local_key)
.expect("Signing libp2p-noise static DH keypair failed."),
)
.multiplex(libp2p_yamux::YamuxConfig::default());

let quic = {
let mut config = libp2p_quic::Config::new(&local_key);
config.support_draft_29 = true;
libp2p_quic::async_std::Transport::new(config)
};

let dns = block_on(DnsConfig::system(OrTransport::new(quic, tcp))).unwrap();

dns.map(|either_output, _| match either_output {
Either::Left((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
Either::Right((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
})
.boxed()
};

let mut swarm = SwarmBuilder::with_async_std_executor(
transport,
libp2p_perf::client::behaviour::Behaviour::default(),
local_peer_id,
)
.substream_upgrade_protocol_override(upgrade::Version::V1Lazy)
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
.build();

swarm.dial(opts.server_address).unwrap();
let server_peer_id = block_on(async {
loop {
match swarm.next().await.unwrap() {
SwarmEvent::ConnectionEstablished { peer_id, .. } => return Ok(peer_id),
SwarmEvent::OutgoingConnectionError { peer_id, error } => {
bail!("Outgoing connection error to {:?}: {:?}", peer_id, error);
}
e => panic!("{e:?}"),
}
}
})?;

swarm.behaviour_mut().perf(
server_peer_id,
RunParams {
to_send: 10 * 1024 * 1024,
to_receive: 10 * 1024 * 1024,
},
)?;

let result = block_on(async {
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
loop {
match swarm.next().await.unwrap() {
SwarmEvent::ConnectionEstablished {
peer_id, endpoint, ..
} => {
info!("Established connection to {:?} via {:?}", peer_id, endpoint);
}
SwarmEvent::OutgoingConnectionError { peer_id, error } => {
info!("Outgoing connection error to {:?}: {:?}", peer_id, error);
}
SwarmEvent::Behaviour(libp2p_perf::client::behaviour::Event { id: _, result }) => {
break result
}
e => panic!("{e:?}"),
}
}
});

let stats = result?;

let sent_mebibytes = stats.params.to_send as f64 / 1024.0 / 1024.0;
let sent_time = (stats.timers.write_done - stats.timers.write_start).as_secs_f64();
let sent_bandwidth_mebibit_second = (sent_mebibytes * 8.0) / sent_time;

let received_mebibytes = stats.params.to_receive as f64 / 1024.0 / 1024.0;
let receive_time = (stats.timers.read_done - stats.timers.write_done).as_secs_f64();
let receive_bandwidth_mebibit_second = (received_mebibytes * 8.0) / receive_time;

println!(
"Finished run: Sent {sent_mebibytes} MiB in {sent_time} s with \
{sent_bandwidth_mebibit_second} MiBit/s and received \
{received_mebibytes} MiB in {receive_time} s with \
{receive_bandwidth_mebibit_second} MiBit/s",
);

Ok(())
}
131 changes: 131 additions & 0 deletions protocols/perf/src/bin/perf-server.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
// Copyright 2023 Protocol Labs.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
// DEALINGS IN THE SOFTWARE.

use clap::Parser;
use futures::{executor::block_on, future::Either, StreamExt};
use libp2p_core::{
identity, muxing::StreamMuxerBox, transport::OrTransport, upgrade, PeerId, Transport,
};
use libp2p_dns::DnsConfig;
use libp2p_swarm::{SwarmBuilder, SwarmEvent};
use log::{error, info};

#[derive(Debug, Parser)]
#[clap(name = "libp2p perf server")]
struct Opts {}

fn main() {
env_logger::init();

let _opts = Opts::parse();

// Create a random PeerId
let local_key = identity::Keypair::generate_ed25519();
let local_peer_id = PeerId::from(local_key.public());
println!("Local peer id: {local_peer_id:?}");
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

let transport = {
let tcp =
libp2p_tcp::async_io::Transport::new(libp2p_tcp::Config::default().port_reuse(true))
.upgrade(upgrade::Version::V1Lazy)
.authenticate(
libp2p_noise::NoiseAuthenticated::xx(&local_key)
.expect("Signing libp2p-noise static DH keypair failed."),
)
.multiplex(libp2p_yamux::YamuxConfig::default());

let quic = {
let mut config = libp2p_quic::Config::new(&local_key);
config.support_draft_29 = true;
libp2p_quic::async_std::Transport::new(config)
};

let dns = block_on(DnsConfig::system(OrTransport::new(quic, tcp))).unwrap();

dns.map(|either_output, _| match either_output {
Either::Left((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
Either::Right((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
})
.boxed()
};

let mut swarm = SwarmBuilder::with_async_std_executor(
transport,
libp2p_perf::server::behaviour::Behaviour::default(),
local_peer_id,
)
.substream_upgrade_protocol_override(upgrade::Version::V1Lazy)
.build();

swarm
.listen_on("/ip4/0.0.0.0/tcp/4001".parse().unwrap())
.unwrap();

swarm
.listen_on("/ip4/0.0.0.0/udp/4001/quic-v1".parse().unwrap())
.unwrap();

block_on(async {
loop {
match swarm.next().await.unwrap() {
SwarmEvent::NewListenAddr { address, .. } => {
info!("Listening on {:?}", address);
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
}
SwarmEvent::IncomingConnection { .. } => {}
e @ SwarmEvent::IncomingConnectionError { .. } => {
error!("{e:?}");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of printing the enum variant, could only print the error, then you wouldn't need to bind the pattern.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then I wouldn't know who (i.e. which dialer) caused the incoming connection error. Am I missing something?

SwarmEvent::ConnectionEstablished {
peer_id, endpoint, ..
} => {
info!("Established connection to {:?} via {:?}", peer_id, endpoint);
}
SwarmEvent::ConnectionClosed { .. } => {}
SwarmEvent::Behaviour(libp2p_perf::server::behaviour::Event::Finished {
remote_peer_id,
stats,
}) => {
let received_mebibytes = stats.params.received as f64 / 1024.0 / 1024.0;
let receive_time =
(stats.timers.read_done - stats.timers.read_start).as_secs_f64();
let receive_bandwidth_mebibit_second =
(received_mebibytes * 8.0) / receive_time;

let sent_mebibytes = stats.params.sent as f64 / 1024.0 / 1024.0;
let sent_time =
(stats.timers.write_done - stats.timers.read_done).as_secs_f64();
let sent_bandwidth_mebibit_second = (sent_mebibytes * 8.0) / sent_time;

info!(
"Finished run with {}: Received {} MiB in {} s with {} MiBit/s and sent {} MiB in {} s with {} MiBit/s",
remote_peer_id,
received_mebibytes,
receive_time,
receive_bandwidth_mebibit_second,
sent_mebibytes,
sent_time,
sent_bandwidth_mebibit_second,
)
}
e => panic!("{e:?}"),
}
}
})
}
Loading