add support for sending error codes on session close #121
Conversation
sukunrt
force-pushed
the
sukun/conn-error-2
branch
from
9d54c73
to
f480b10
Compare
sukunrt
force-pushed
the
sukun/conn-error-2
branch
from
f480b10
to
efa52bc
Compare
sukunrt
changed the title
sukunrt
force-pushed
the
sukun/conn-error-2
branch
4 times, most recently
from
af1ac71
to
18a75f1
Compare
|
Is there an issue describing how this will be used? I usually want to send errors when closing a stream, less when closing a connection. Is the plan to use this in the connection manager? |
|
Apologies! The specs are here: libp2p/specs#623 In go-libp2p we will mostly use Connection Close error codes from the connection manager. Applications can define their error codes. Stream error codes will be introduced in a separate PR. |
sukunrt
force-pushed
the
sukun/conn-error-2
branch
from
18a75f1
to
4b262c0
Compare
sukunrt
force-pushed
the
sukun/conn-error-2
branch
from
4aed194
to
8adb9a8
Compare
|
Oh, I see. |
| // Attempts to send a GoAway before closing the connection. The GoAway may not actually be sent depending on the | ||
| // semantics of the underlying net.Conn. For TCP connections, it may be dropped depending on LINGER value or | ||
| // if there's unread data in the kernel receive buffer. |
There was a problem hiding this comment.
Why send an error in this case? I'm concerned that we're changing the semantics to block for up to 15 seconds.
| // The GoAway may not actually be sent depending on the semantics of the underlying net.Conn. | ||
| // For TCP connections, it may be dropped depending on LINGER value or if there's unread data in the kernel | ||
| // receive buffer. | ||
| func (s *Session) CloseWithError(errCode uint32) error { |
There was a problem hiding this comment.
Have we updated the connection manager to be able to deal with potential blocking here? Also, we should probably document it.
const.go
Outdated
| @@ -117,6 +152,7 @@ const ( | |||
| // It's not an implementation choice, the value defined in the specification. | |||
| initialStreamWindow = 256 * 1024 | |||
| maxStreamWindow = 16 * 1024 * 1024 | |||
| goAwayWaitTime = 5 * time.Second | |||
There was a problem hiding this comment.
I'd make this_much shorter. E.g., 100ms. We shouldn't need to wait on a round-trip here.
|
Blocking on a connection is unfortunate. The "correct" way here would be to send the error code with the RST packet. The latest TCP RFC also recommends this: https://www.rfc-editor.org/rfc/rfc9293.html#name-reset-processing
But no implementation provides this API at the moment. |
We can consider running this Async in a different goroutine. That'll use more memory and mess up the resource manager accounting for a short duration, but it'll cause less issues with existing code. The current implementation wouldn't block in most of the cases. If there's receive window available at the remote end, it wont block. |
|
So, we usually only close connections from the connection manager, right? IMO, we should consider:
On the other hand.... spawning a goroutine isn't terrible. The resource consumption is fairly minimal in modern go and blocking will tie up resources just the same. |
|
Actually... we already have a goroutine. Can we reuse it? Are we not accounting for that one in the resource manager? |
A common case will also be the resource manager closing connections after seeing some limit has been reached. |
This adds support for sending error codes on closing a session.
The error isn't guaranteed to be sent to remote. It depends on the LINGER value for the TCP socket and also on whether there's any unread data in the receive buffer when close was called. In both these situations, the GoAway frame might be dropped.
To reliably send error codes, we'd have to send a TCP FIN packet and wait for remote its half of the connection. This would also require sending everything that's pending in the kernel write buffer. To not introduce this 1RTT delay of closing, I've opted to make this a best effort implementation.