fix(dcutr): always check for relayed connection first #3982

arpankapoor · 2023-05-24T16:59:48Z

Description

For a non-relay connection, check if it corresponds to a direct connection upgrade.

As discussed in #3964.

Notes & open questions

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
A changelog entry has been made in the appropriate crates

kpp · 2023-05-24T18:07:17Z

Is it okay that sometimes QUIC client cannot connect to a server with DCUtR example?

BehaviourEvent: CircuitReqDenied { src_peer_id: PeerId("12D3KooWQYhTNQdmr3ArTeUHRYzFg94BKyTkoWBDWez9kSCVe2Xo"), dst_peer_id: PeerId("12D3KooWH3uVF6wv47WnArKHk5p6cvgCJEb74UTmxztmQDc298L3") }

thomaseizinger

Thanks!

protocols/dcutr/src/behaviour_impl.rs

arpankapoor · 2023-05-25T02:34:33Z

Is it okay that sometimes QUIC client cannot connect to a server with DCUtR example?
BehaviourEvent: CircuitReqDenied { src_peer_id: PeerId("12D3KooWQYhTNQdmr3ArTeUHRYzFg94BKyTkoWBDWez9kSCVe2Xo"), dst_peer_id: PeerId("12D3KooWH3uVF6wv47WnArKHk5p6cvgCJEb74UTmxztmQDc298L3") }

Do you suspect this issue to be due to this change?

mxinden

@arpankapoor what problem is this pull request solving? Why is it relevant to first check whether the connection is relayed. Isn't first checking self.outgoing_direct_connection_attempts and then is_relayed just the same?

protocols/dcutr/src/behaviour_impl.rs

arpankapoor · 2023-05-25T07:49:39Z

@arpankapoor what problem is this pull request solving? Why is it relevant to first check whether the connection is relayed. Isn't first checking self.outgoing_direct_connection_attempts and then is_relayed just the same?

The current version of the code returns a dummy handler for direct connections corresponding to a relayed connection. This is because outgoing_direct_connection_attempts has the relayed connection id in its key and not the direct connection id. Essentially, the Some(_) arm of the match will never be hit.

Related comment.

thomaseizinger · 2023-05-25T08:13:12Z

@arpankapoor what problem is this pull request solving? Why is it relevant to first check whether the connection is relayed. Isn't first checking self.outgoing_direct_connection_attempts and then is_relayed just the same?

The current version of the code returns a dummy handler for direct connections corresponding to a relayed connection. This is because outgoing_direct_connection_attempts has the relayed connection id in its key and not the direct connection id. Essentially, the Some(_) arm of the match will never be hit.

Related comment.

I just dug into this a bit more and the reason our tests are passing this is because we only assert that the direct connection is established but we don't assert that we report the correct event (which is only reported by the direct::Handler).

thomaseizinger

I think this can actually be simplified a lot further, please see comments!

protocols/dcutr/src/behaviour_impl.rs

thomaseizinger · 2023-05-25T08:25:37Z

protocols/dcutr/src/behaviour_impl.rs

+            ))); // TODO: We could make two `handler::relayed::Handler` here, one inbound one outbound.
+        }
+
+        if let Some(&relayed_connection_id) = self.direct_to_relayed_connections.get(&connection_id)


I don't think this can ever be hit actually. In hole-punching, we cannot for sure tell which inbound connection was the one that the remote achieved through hole punching.

Plus, all this state tracking here uses ConnectionIds that are created as part of dialing, i.e. outbound connections.

So unless I am missing something, this function can be reduced to always return a dummy::ConnectionHandler for non-relayed connections.

@mxinden Thoughts?

I think I agree with your assessment. I'll make this change after @mxinden's comments.

You are right @thomaseizinger.

For what it is worth, likely my train of thought that ended up with the code above:

In NetworkBehaviour::on_connection_handler_event on OutboundConnectNegotiated we trigger a Dial with role_override() (i.e. as_listener) and then insert the maybe_direct_connection_id from the DialOpts into self.direct_to_relayed_connections.

rust-libp2p/protocols/dcutr/src/behaviour_impl.rs

Lines 379 to 395 in d4c4078

Either::Left(handler::relayed::Event::OutboundConnectNegotiated { remote_addrs }) => {

let opts = DialOpts::peer_id(event_source)

.condition(dial_opts::PeerCondition::Always)

.addresses(remote_addrs)

.override_role()

.build();

let maybe_direct_connection_id = opts.connection_id();

self.direct_to_relayed_connections

.insert(maybe_direct_connection_id, relayed_connection_id);

*self

.outgoing_direct_connection_attempts

.entry((relayed_connection_id, event_source))

.or_default() += 1;

self.queued_events.push_back(ToSwarm::Dial { opts });

}

On success this would be reported to the Behaviour via NetworkBehaviour::handle_established_outbound_connection and not as assumed here in NetworkBehaviour::handle_established_inbound_connection.

How about adding an assert here, that connection_id is not in self.direct_to_relayed_connections?

…removal

mxinden

Overall this looks good to me. Thank you for the work here. I find this domain very tricky.

mxinden · 2023-05-28T04:50:18Z

protocols/dcutr/src/behaviour_impl.rs

+            ))); // TODO: We could make two `handler::relayed::Handler` here, one inbound one outbound.
+        }
+
+        if let Some(&relayed_connection_id) = self.direct_to_relayed_connections.get(&connection_id)


You are right @thomaseizinger.

For what it is worth, likely my train of thought that ended up with the code above:

In NetworkBehaviour::on_connection_handler_event on OutboundConnectNegotiated we trigger a Dial with role_override() (i.e. as_listener) and then insert the maybe_direct_connection_id from the DialOpts into self.direct_to_relayed_connections.

rust-libp2p/protocols/dcutr/src/behaviour_impl.rs

Lines 379 to 395 in d4c4078

Either::Left(handler::relayed::Event::OutboundConnectNegotiated { remote_addrs }) => {

let opts = DialOpts::peer_id(event_source)

.condition(dial_opts::PeerCondition::Always)

.addresses(remote_addrs)

.override_role()

.build();

let maybe_direct_connection_id = opts.connection_id();

self.direct_to_relayed_connections

.insert(maybe_direct_connection_id, relayed_connection_id);

*self

.outgoing_direct_connection_attempts

.entry((relayed_connection_id, event_source))

.or_default() += 1;

self.queued_events.push_back(ToSwarm::Dial { opts });

}

On success this would be reported to the Behaviour via NetworkBehaviour::handle_established_outbound_connection and not as assumed here in NetworkBehaviour::handle_established_inbound_connection.

mxinden · 2023-05-28T04:51:12Z

protocols/dcutr/src/behaviour_impl.rs

+            ))); // TODO: We could make two `handler::relayed::Handler` here, one inbound one outbound.
+        }
+
+        if let Some(&relayed_connection_id) = self.direct_to_relayed_connections.get(&connection_id)


How about adding an assert here, that connection_id is not in self.direct_to_relayed_connections?

protocols/dcutr/src/behaviour_impl.rs

arpankapoor · 2023-05-29T07:20:21Z

Overall this looks good to me. Thank you for the work here. I find this domain very tricky.

I've applied the changes you suggested. Please let me know if you'd like anything else updated/changed.

mxinden

This looks good to me. Thanks @arpankapoor for the work.

Will give @thomaseizinger a chance to review as well.

protocols/dcutr/src/behaviour_impl.rs

mxinden · 2023-05-30T02:06:45Z

Moving an out-of-band discussion here:

any chance you can consider [this pull request] for the upcoming release?

I am fairly certain, unless @thomaseizinger raises any larger objections, that this pull request will make it into libp2p v0.52.0.

This is a bug fix and it does not break our public API. Thus, in case you want this before libp2p v0.52.0, we could as well include it as a patch release on top of libp2p v0.51. If so, we can backport to branch v0.51 once this pull request is merged.

mxinden · 2023-05-30T10:46:32Z

protocols/dcutr/src/behaviour_impl.rs

+            return Ok(Either::Right(Either::Left(
+                handler::direct::Handler::default(),
+            )));


With recent refactorings in libp2p-swarm, there is actually no more need for handler::direct::Handler at all. The only thing that it does is report an event to the behaviour that is then bubbled to the user. The behaviour could as well just bubble the event to the user here already and return just a dummy handler.

The above improvement is only slightly related to this pull request. In case you want to tackle it @arpankapoor in this pull request, that would be much appreciated. Though I am happy to merge as is and tackle it some other time.

Idea raised by @thomaseizinger in a synchronous discussion just now.

I am in favor of that refactoring but no need to include in this PR.

Yeah, it could be tackled in another PR perhaps.

Thanks! I'll merge this as is then :)

Tracked here #4013. Also unresolving here for easier discovery in the future.

Fix DCUtR NetworkBehavior's established connection handler

4ed1427

arpankapoor mentioned this pull request May 24, 2023

feat(quic): implement hole punching #3964

Merged

4 tasks

thomaseizinger reviewed May 24, 2023

View reviewed changes

protocols/dcutr/src/behaviour_impl.rs Outdated Show resolved Hide resolved

thomaseizinger changed the title ~~fix(dcutr): Fix DCUtR NetworkBehavior's established connection handler~~ fix(dcutr): always check for relayed connection first May 24, 2023

mxinden reviewed May 25, 2023

View reviewed changes

protocols/dcutr/src/behaviour_impl.rs Outdated Show resolved Hide resolved

Use early return

bf1dfce

thomaseizinger reviewed May 25, 2023

View reviewed changes

arpankapoor added 4 commits May 25, 2023 17:59

Assert that outgoing_direct_connection_attempts entry was removed

770e9d0

Fix failing test

527371e

Add back removal of outgoing_direct_connection_attempts entry

3b444eb

Add assert on Listener for outgoing_direct_connection_attempts entry …

5e9e0a3

…removal

mxinden reviewed May 28, 2023

View reviewed changes

mxinden mentioned this pull request May 29, 2023

feat(dcutr): keep connection alive while we are using it #3960

Merged

4 tasks

Return dummy handler directly in inbound handler

d1aba5f

Fix clippy warning

b3c0996

mxinden approved these changes May 30, 2023

View reviewed changes

protocols/dcutr/src/behaviour_impl.rs Show resolved Hide resolved

protocols/dcutr/src/behaviour_impl.rs Show resolved Hide resolved

Add comment

5948aee

mxinden reviewed May 30, 2023

View reviewed changes

thomaseizinger approved these changes May 30, 2023

View reviewed changes

thomaseizinger added the send-it label May 30, 2023

thomaseizinger and others added 2 commits May 30, 2023 19:58

Merge branch 'master' into dcutr-fix

981735b

Merge branch 'master' into dcutr-fix

bf8d1b6

mergify bot merged commit 5321a44 into libp2p:master May 31, 2023

mxinden mentioned this pull request May 31, 2023

fix(dcutr): remove direct handler #4013

Closed

mxinden mentioned this pull request Jun 1, 2023

dialing client raised Connection closed with error NoAddresses, relay server log is no protocol could be agreed upon for inbound stream #4007

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(dcutr): always check for relayed connection first #3982

fix(dcutr): always check for relayed connection first #3982

arpankapoor commented May 24, 2023

kpp commented May 24, 2023

thomaseizinger left a comment

arpankapoor commented May 25, 2023

mxinden left a comment

arpankapoor commented May 25, 2023

thomaseizinger commented May 25, 2023

thomaseizinger left a comment

thomaseizinger May 25, 2023

arpankapoor May 25, 2023

mxinden May 28, 2023

mxinden May 28, 2023

mxinden left a comment

mxinden May 28, 2023

mxinden May 28, 2023

arpankapoor commented May 29, 2023

mxinden left a comment

mxinden commented May 30, 2023

mxinden May 30, 2023

thomaseizinger May 30, 2023

arpankapoor May 30, 2023

thomaseizinger May 30, 2023

mxinden May 31, 2023

	Either::Left(handler::relayed::Event::OutboundConnectNegotiated { remote_addrs }) => {
	let opts = DialOpts::peer_id(event_source)
	.condition(dial_opts::PeerCondition::Always)
	.addresses(remote_addrs)
	.override_role()
	.build();

	let maybe_direct_connection_id = opts.connection_id();

	self.direct_to_relayed_connections
	.insert(maybe_direct_connection_id, relayed_connection_id);
	*self
	.outgoing_direct_connection_attempts
	.entry((relayed_connection_id, event_source))
	.or_default() += 1;
	self.queued_events.push_back(ToSwarm::Dial { opts });
	}

fix(dcutr): always check for relayed connection first #3982

fix(dcutr): always check for relayed connection first #3982

Conversation

arpankapoor commented May 24, 2023

Description

Notes & open questions

Change checklist

kpp commented May 24, 2023

thomaseizinger left a comment

Choose a reason for hiding this comment

arpankapoor commented May 25, 2023

mxinden left a comment

Choose a reason for hiding this comment

arpankapoor commented May 25, 2023

thomaseizinger commented May 25, 2023

thomaseizinger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mxinden left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arpankapoor commented May 29, 2023

mxinden left a comment

Choose a reason for hiding this comment

mxinden commented May 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment