Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add Experimental QUIC support #4577

Closed
wants to merge 88 commits into from
Closed
Changes from 5 commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
61d4ae1
Initial quic support
AgeManning Aug 2, 2023
3eb4937
enable Quic transport on libp2p
jxs Aug 2, 2023
0ebdc79
allow support for the two different UDP ports,
jxs Aug 2, 2023
1344786
Merge pull request #5 from jxs/quic
AgeManning Aug 3, 2023
29db04d
Add ENR structure for quick and specify listening criteria
AgeManning Aug 3, 2023
5261b86
discovery: append quic multiaddrs along with tcp,
jxs Aug 4, 2023
7305595
Add some documentation for protocol developers
AgeManning Aug 7, 2023
6c97c5a
discovery return Enr's instead of PeerId's
jxs Aug 7, 2023
cd66aee
isolate multiaddresses gathering to dial on PeerManager
jxs Aug 7, 2023
f98f143
Merge pull request #6 from jxs/quic
AgeManning Aug 8, 2023
d8bb593
Small logging and comment changes
AgeManning Aug 8, 2023
d931d2a
Merge latest unstable
AgeManning Aug 8, 2023
4c9159c
Correct line spacing in book docs
AgeManning Aug 8, 2023
3492646
Appease the all-wise all-knowing clippy
AgeManning Aug 8, 2023
c4f0a38
fmt
AgeManning Aug 8, 2023
583c233
Correct quic port and improve logs
AgeManning Aug 8, 2023
1821941
Network: cleanup remove no longer required fn
jxs Aug 8, 2023
ae54809
Discovery:
jxs Aug 8, 2023
191e0ae
PeerManager: move should_dial() logic to dial_peer()
jxs Aug 8, 2023
31f45b7
cargo fmt
jxs Aug 8, 2023
d3953e0
fmt
AgeManning Aug 9, 2023
dcc78cc
Remove unused var
AgeManning Aug 9, 2023
c7c1749
Can't change a single line without fmt complaining
AgeManning Aug 9, 2023
14b42c1
Add QUIC to default ENR builder and fight tests
AgeManning Aug 10, 2023
0ff3c92
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Aug 10, 2023
632a57b
cargo fmt
jxs Aug 10, 2023
2def400
remove leftover println's
jxs Aug 10, 2023
7de61b5
fix benchmarks
jxs Aug 10, 2023
4771634
fix debug tests
jxs Aug 10, 2023
8858f0e
Correct port mapping bug
AgeManning Aug 11, 2023
8dfeb3c
Merge latest unstable
AgeManning Aug 11, 2023
5b5b262
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Aug 11, 2023
2fbd384
Merge pull request #7 from jxs/quic
AgeManning Aug 11, 2023
ee67351
fix enr-quic6-port test
jxs Aug 11, 2023
4c5cffc
update libp2p-quic to 0.9.2
jxs Aug 12, 2023
68d3a84
cargo clippy
jxs Aug 12, 2023
fbdd015
EnrExt: rename quic to quic4 for consistency
jxs Aug 12, 2023
a10019a
increase quic port on dht persistance test
jxs Aug 12, 2023
28b4e06
Merge branch 'unstable' of https://github.com/sigp/lighthouse into quic
jxs Aug 14, 2023
6498908
fix quic port on unused_v6_ports test
jxs Aug 14, 2023
91c4fad
make rpc tests use tcp multiaddr,
jxs Aug 14, 2023
ce67467
fix network_port_discovery_quic_port_flags_over_ipv4_and_ipv6 test
jxs Aug 14, 2023
3527d91
fix test
jxs Aug 16, 2023
137bb89
add quic to initialized ENR log
jxs Aug 16, 2023
99f72d0
add quic to ENR echo logs
jxs Aug 16, 2023
adfb7db
fix simulations by incrementing quic port by 1000
jxs Aug 17, 2023
ec31927
Merge branch 'unstable' of https://github.com/sigp/lighthouse into quic
jxs Aug 17, 2023
4adc916
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Aug 17, 2023
3d3d8c8
add quic rpc tests.
jxs Aug 20, 2023
1bb942b
put quic transport in the dns transport
jxs Aug 23, 2023
5c29f83
add multiaddress to connection established logging
jxs Aug 23, 2023
262b1e5
update local test net scripts for quic
jxs Aug 24, 2023
3cd11e6
Merge branch 'unstable' of https://github.com/sigp/lighthouse into quic
jxs Aug 25, 2023
ac83825
update libp2p-quic to stable
jxs Aug 25, 2023
9247496
add connected TCP and QUIC peers to prometheus metrics
jxs Aug 25, 2023
6236391
Differentiate the metrics in prometheus
AgeManning Aug 28, 2023
19f5e7e
Merge branch 'unstable' of https://github.com/sigp/lighthouse into quic
jxs Aug 28, 2023
13355bc
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Aug 28, 2023
e6294c5
cargo fmt
jxs Aug 28, 2023
96a3810
remove unrequired features section
jxs Aug 28, 2023
a8ab8c7
move parse_listening_addresses for better reviewing
jxs Aug 28, 2023
82f13bc
improve doc wording
jxs Aug 28, 2023
36a044a
fix metrics protocol identification
jxs Aug 28, 2023
9f74dd9
update socket address for quic connections,
jxs Aug 30, 2023
c14b845
address review
jxs Aug 31, 2023
b90e969
Remove references and assumptions to TCP based libp2p port
AgeManning Sep 4, 2023
aaced27
fmt
AgeManning Sep 4, 2023
f2cd1b1
Merge branch 'unstable' of https://github.com/sigp/lighthouse into quic
jxs Sep 4, 2023
5d0dee5
Clippy lints
AgeManning Sep 5, 2023
1e838a1
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Sep 5, 2023
655a757
Improve inbound connection log
AgeManning Sep 5, 2023
ed3dfb6
Update QUIC ports on UPnP mapping changes
AgeManning Sep 6, 2023
61715fe
Merge latest unstable
AgeManning Sep 7, 2023
19f3c46
fix quic metric protocol on connection close
jxs Sep 7, 2023
84cff62
Merge branch 'quic' of github.com:AgeManning/lighthouse into quic
jxs Sep 7, 2023
28d81d6
fix doppelganger_protection script
jxs Sep 7, 2023
e230a57
fix ports for doppelganger protection test on ci
jxs Sep 7, 2023
dd44476
fix missing port for last validator client
jxs Sep 7, 2023
bba8220
fix last two ports in doppelganger protection test
jxs Sep 8, 2023
df64302
fix port order on start_local_testnet.sh
jxs Sep 8, 2023
2be25d4
Hopefully improve port management in rpc tests
AgeManning Sep 11, 2023
d01b2e1
fmt
AgeManning Sep 11, 2023
3ac2513
Merge latest unstable
AgeManning Sep 11, 2023
74f0375
Update tests due to unstable merge
AgeManning Sep 11, 2023
fa11ff9
Small test changes
AgeManning Sep 12, 2023
231e7db
Nuke windows self-hosted runner
AgeManning Sep 13, 2023
a4faad5
Revert "Nuke windows self-hosted runner"
AgeManning Sep 13, 2023
41c8207
beacon node: move connected peer metrics,
jxs Sep 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions beacon_node/lighthouse_network/src/config.rs
Original file line number Diff line number Diff line change
@@ -58,18 +58,24 @@ pub struct Config {
/// that no discovery address has been set in the CLI args.
pub enr_address: (Option<Ipv4Addr>, Option<Ipv6Addr>),

/// The disc4 port to broadcast to peers in order to reach back for discovery.
pub enr_disc4_port: Option<u16>,
/// The udp port to broadcast to peers in order to reach back for discovery.
pub enr_udp4_port: Option<u16>,

/// The UDP port to broadcast to peers in order to reach back for quic libp2p services.
pub enr_quic4_port: Option<u16>,

/// The tcp4 port to broadcast to peers in order to reach back for libp2p services.
pub enr_tcp4_port: Option<u16>,

/// The disc6 port to broadcast to peers in order to reach back for discovery.
pub enr_disc6_port: Option<u16>,
/// The udp6 port to broadcast to peers in order to reach back for discovery.
pub enr_udp6_port: Option<u16>,

/// The tcp6 port to broadcast to peers in order to reach back for libp2p services.
pub enr_tcp6_port: Option<u16>,

/// The UDP port to broadcast to peers in order to reach back for quic libp2p services.
pub enr_quic6_port: Option<u16>,

/// Target number of connected peers.
pub target_peers: usize,

@@ -333,10 +339,11 @@ impl Default for Config {
network_dir,
listen_addresses,
enr_address: (None, None),

enr_disc4_port: None,
enr_udp4_port: None,
enr_quic4_port: None,
enr_tcp4_port: None,
enr_disc6_port: None,
enr_udp6_port: None,
enr_quic6_port: None,
enr_tcp6_port: None,
target_peers: 50,
gs_config,
@@ -591,4 +598,4 @@ pub const fn is_global_ipv6(addr: &Ipv6Addr) -> bool {
|| is_documentation(addr)
|| is_unique_local(addr)
|| is_unicast_link_local(addr))
}
}
35 changes: 30 additions & 5 deletions beacon_node/lighthouse_network/src/discovery/enr.rs
Original file line number Diff line number Diff line change
@@ -17,6 +17,8 @@ use std::path::Path;
use std::str::FromStr;
use types::{EnrForkId, EthSpec};

use super::enr_ext::{EnrExt, QUIC6_ENR_KEY, QUIC_ENR_KEY};

/// The ENR field specifying the fork id.
pub const ETH2_ENR_KEY: &str = "eth2";
/// The ENR field specifying the attestation subnet bitfield.
@@ -142,7 +144,7 @@ pub fn build_or_load_enr<T: EthSpec>(

pub fn create_enr_builder_from_config<T: EnrKey>(
config: &NetworkConfig,
enable_tcp: bool,
enable_libp2p: bool,
) -> EnrBuilder<T> {
let mut builder = EnrBuilder::new("v4");
let (maybe_ipv4_address, maybe_ipv6_address) = &config.enr_address;
@@ -155,15 +157,35 @@ pub fn create_enr_builder_from_config<T: EnrKey>(
builder.ip6(*ip);
}

if let Some(udp4_port) = config.enr_disc4_port {
if let Some(udp4_port) = config.enr_udp4_port {
builder.udp4(udp4_port);
}

if let Some(udp6_port) = config.enr_disc6_port {
if let Some(udp6_port) = config.enr_udp6_port {
builder.udp6(udp6_port);
}

if enable_tcp {
// Add QUIC fields to the ENR.
// If `enable_libp2p` is disabled, then we should not support QUIC in the ENR either.
if enable_libp2p && !config.disable_quic_support {
// If we are listening on ipv4, add the quic ipv4 port
if let Some(quic4_port) = config
.enr_quic4_port
.or_else(|| config.listen_addrs().v4().map(|v4_addr| v4_addr.quic_port))
{
builder.add_value(QUIC_ENR_KEY, &quic4_port);
}

// If we are listening on ipv6, add the quic ipv6 port
if let Some(quic6_port) = config
.enr_quic6_port
.or_else(|| config.listen_addrs().v6().map(|v6_addr| v6_addr.quic_port))
{
builder.add_value(QUIC6_ENR_KEY, &quic6_port);
}
}

if enable_libp2p {
// If the ENR port is not set, and we are listening over that ip version, use the listening port instead.
let tcp4_port = config
.enr_tcp4_port
@@ -218,6 +240,9 @@ fn compare_enr(local_enr: &Enr, disk_enr: &Enr) -> bool {
// tcp ports must match
&& local_enr.tcp4() == disk_enr.tcp4()
&& local_enr.tcp6() == disk_enr.tcp6()
// quic ports must match
&& local_enr.quic() == disk_enr.quic()
&& local_enr.quic6() == disk_enr.quic6()
// must match on the same fork
&& local_enr.get(ETH2_ENR_KEY) == disk_enr.get(ETH2_ENR_KEY)
// take preference over disk udp port if one is not specified
@@ -258,4 +283,4 @@ pub fn save_enr_to_disk(dir: &Path, enr: &Enr, log: &slog::Logger) {
);
}
}
}
}
6 changes: 3 additions & 3 deletions beacon_node/lighthouse_network/src/discovery/enr_ext.rs
Original file line number Diff line number Diff line change
@@ -6,8 +6,8 @@ use libp2p::core::multiaddr::Protocol;
use libp2p::identity::{ed25519, secp256k1, KeyType, Keypair, PublicKey};
use tiny_keccak::{Hasher, Keccak};

const QUIC_ENR_KEY: &str = "quic";
const QUIC6_ENR_KEY: &str = "quic6";
pub const QUIC_ENR_KEY: &str = "quic";
pub const QUIC6_ENR_KEY: &str = "quic6";

/// Extend ENR for libp2p types.
pub trait EnrExt {
@@ -397,4 +397,4 @@ mod tests {

assert_eq!(enr.node_id(), node_id);
}
}
}
21 changes: 13 additions & 8 deletions beacon_node/lighthouse_network/src/listen_addr.rs
Original file line number Diff line number Diff line change
@@ -60,26 +60,31 @@ impl ListenAddress {

/// Returns the addresses the Swarm will listen on, given the setup.
pub fn listen_addresses(&self) -> impl Iterator<Item = Multiaddr> {
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
let v4_tcp_multiaddrs = self
let v4_tcp_multiaddr = self
.v4()
.map(|v4_addr| Multiaddr::from(v4_addr.addr).with(Protocol::Tcp(v4_addr.tcp_port)));

let v4_quic_multiaddrs = self.v4().map(|v4_addr| {
let v4_quic_multiaddr = self.v4().map(|v4_addr| {
Multiaddr::from(v4_addr.addr)
.with(Protocol::Udp(v4_addr.quic_port))
.with(Protocol::QuicV1)
});

let v6_tcp_multiaddrs = self
let v6_quic_multiaddr = self.v6().map(|v6_addr| {
Multiaddr::from(v6_addr.addr)
.with(Protocol::Udp(v6_addr.quic_port))
.with(Protocol::QuicV1)
});

let v6_tcp_multiaddr = self
.v6()
.map(|v6_addr| Multiaddr::from(v6_addr.addr).with(Protocol::Tcp(v6_addr.tcp_port)));

// TODO: Add QUIC IPv6 multiaddr once it is supported

v4_tcp_multiaddrs
v4_tcp_multiaddr
.into_iter()
.chain(v4_quic_multiaddrs)
.chain(v6_tcp_multiaddrs)
.chain(v4_quic_multiaddr)
.chain(v6_quic_multiaddr)
.chain(v6_tcp_multiaddr)
}

#[cfg(test)]
26 changes: 21 additions & 5 deletions beacon_node/src/cli.rs
Original file line number Diff line number Diff line change
@@ -175,23 +175,39 @@ pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
.long("enr-udp-port")
.value_name("PORT")
.help("The UDP4 port of the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IpV4.")
can connect to your local node on this port over IPv4.")
.takes_value(true),
)
.arg(
Arg::with_name("enr-quic-port")
.long("enr-quic-port")
.value_name("PORT")
.help("The quic UDP4 port that will be set on the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IPv4.")
.takes_value(true),
)
.arg(
Arg::with_name("enr-udp6-port")
.long("enr-udp6-port")
.value_name("PORT")
.help("The UDP6 port of the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IpV6.")
can connect to your local node on this port over IPv6.")
.takes_value(true),
)
.arg(
Arg::with_name("enr-quic6-port")
.long("enr-quic6-port")
.value_name("PORT")
.help("The quic UDP6 port that will be set on the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IPv6.")
.takes_value(true),
)
.arg(
Arg::with_name("enr-tcp-port")
.long("enr-tcp-port")
.value_name("PORT")
.help("The TCP4 port of the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IpV4. The --port flag is \
can connect to your local node on this port over IPv4. The --port flag is \
used if this is not set.")
.takes_value(true),
)
@@ -200,7 +216,7 @@ pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
.long("enr-tcp6-port")
.value_name("PORT")
.help("The TCP6 port of the local ENR. Set this only if you are sure other nodes \
can connect to your local node on this port over IpV6. The --port6 flag is \
can connect to your local node on this port over IPv6. The --port6 flag is \
used if this is not set.")
.takes_value(true),
)
@@ -1166,4 +1182,4 @@ pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
.takes_value(true)
.possible_values(ProgressiveBalancesMode::VARIANTS)
)
}
}
41 changes: 27 additions & 14 deletions beacon_node/src/config.rs
Original file line number Diff line number Diff line change
@@ -77,6 +77,7 @@ pub fn get_config<E: EthSpec>(

let data_dir_ref = client_config.data_dir().clone();

println!("SET NETWORK CONFIG");
set_network_config(&mut client_config.network, cli_args, &data_dir_ref, log)?;

/*
@@ -916,13 +917,21 @@ pub fn set_network_config(
}

if let Some(enr_udp_port_str) = cli_args.value_of("enr-udp-port") {
config.enr_disc4_port = Some(
config.enr_udp4_port = Some(
enr_udp_port_str
.parse::<u16>()
.map_err(|_| format!("Invalid discovery port: {}", enr_udp_port_str))?,
);
}

if let Some(enr_quic_port_str) = cli_args.value_of("enr-quic-port") {
config.enr_quic4_port = Some(
enr_quic_port_str
.parse::<u16>()
.map_err(|_| format!("Invalid quic port: {}", enr_quic_port_str))?,
);
}

if let Some(enr_tcp_port_str) = cli_args.value_of("enr-tcp-port") {
config.enr_tcp4_port = Some(
enr_tcp_port_str
@@ -932,13 +941,21 @@ pub fn set_network_config(
}

if let Some(enr_udp_port_str) = cli_args.value_of("enr-udp6-port") {
config.enr_disc6_port = Some(
config.enr_udp6_port = Some(
enr_udp_port_str
.parse::<u16>()
.map_err(|_| format!("Invalid discovery port: {}", enr_udp_port_str))?,
);
}

if let Some(enr_quic_port_str) = cli_args.value_of("enr-quic6-port") {
config.enr_quic6_port = Some(
enr_quic_port_str
.parse::<u16>()
.map_err(|_| format!("Invalid quic port: {}", enr_quic_port_str))?,
);
}

if let Some(enr_tcp_port_str) = cli_args.value_of("enr-tcp6-port") {
config.enr_tcp6_port = Some(
enr_tcp_port_str
@@ -958,7 +975,7 @@ pub fn set_network_config(
ipv4_addr.addr
};
config.enr_address.0 = Some(ipv4_enr_addr);
config.enr_disc4_port = Some(ipv4_addr.disc_port);
config.enr_udp4_port = Some(ipv4_addr.disc_port);
}

if let Some(ipv6_addr) = config.listen_addrs().v6().cloned() {
@@ -968,7 +985,7 @@ pub fn set_network_config(
ipv6_addr.addr
};
config.enr_address.1 = Some(ipv6_enr_addr);
config.enr_disc6_port = Some(ipv6_addr.disc_port);
config.enr_udp6_port = Some(ipv6_addr.disc_port);
}
}

@@ -1132,6 +1149,7 @@ pub fn parse_listening_addresses(
cli_args: &ArgMatches,
log: &Logger,
) -> Result<ListenAddress, String> {
dbg!("PARSE");
let listen_addresses_str = cli_args
.values_of("listen-address")
.expect("--listen_addresses has a default value");
@@ -1215,6 +1233,8 @@ pub fn parse_listening_addresses(
format!("Failed to parse --quic6-port as an integer: {parse_error}")
})?;

println!("{:?}", maybe_disc6_port);
println!("{:?}", (maybe_ipv4, maybe_ipv6));
// Now put everything together
let listening_addresses = match (maybe_ipv4, maybe_ipv6) {
(None, None) => {
@@ -1237,18 +1257,9 @@ pub fn parse_listening_addresses(
warn!(log, "When listening only over IPv6, use the --discovery-port flag. The value of --discovery-port6 will be ignored.")
}

// TODO: Remove this warning once https://github.com/libp2p/rust-libp2p/issues/4165
// is resolved.
if maybe_quic_port.is_some() || maybe_quic6_port.is_some() {
warn!(log, "QUIC is currently disabled over Ipv6.")
}

// TODO: Once QUIC is supported over IPv6, uncomment this.
/*
if maybe_quic6_port.is_some() {
warn!(log, "When listening only over IPv6, use the --quic-port flag. The value of --quic-port6 will be ignored.")
}
*/

// use zero ports if required. If not, use the specific udp port. If none given, use
// the tcp port.
@@ -1322,11 +1333,13 @@ pub fn parse_listening_addresses(
.then(unused_port::unused_tcp6_port)
.transpose()?
.unwrap_or(port6);
println!("{:?}", maybe_disc6_port);
let ipv6_disc_port = use_zero_ports
.then(unused_port::unused_udp6_port)
.transpose()?
.or(maybe_disc6_port)
.unwrap_or(ipv6_tcp_port);
println!("{:?}", ipv6_disc_port);
let ipv6_quic_port = use_zero_ports
.then(unused_port::unused_udp6_port)
.transpose()?
@@ -1425,4 +1438,4 @@ where
.into_iter()
.next()
.ok_or(format!("Must provide at least one value to {}", flag_name))
}
}
10 changes: 5 additions & 5 deletions boot_node/src/config.rs
Original file line number Diff line number Diff line change
@@ -59,17 +59,17 @@ impl<T: EthSpec> BootNodeConfig<T> {

// Set the Enr Discovery ports to the listening ports if not present.
if let Some(listening_addr_v4) = network_config.listen_addrs().v4() {
network_config.enr_disc4_port = Some(
network_config.enr_udp4_port = Some(
network_config
.enr_disc4_port
.enr_udp4_port
.unwrap_or(listening_addr_v4.disc_port),
)
};

if let Some(listening_addr_v6) = network_config.listen_addrs().v6() {
network_config.enr_disc6_port = Some(
network_config.enr_udp6_port = Some(
network_config
.enr_disc6_port
.enr_udp6_port
.unwrap_or(listening_addr_v6.disc_port),
)
};
@@ -190,4 +190,4 @@ impl BootNodeConfigSerialization {
enable_enr_auto_update: discv5_config.enr_update,
}
}
}
}
Loading