[multistream-select] Reduce roundtrips in protocol negotiation.

[romanb]

Overview & Motivation

This is another attempt at libp2p#659, superseding libp2p#1121 and sitting on top of libp2p#1203. The following are the main conceptual changes:

1. The multistream-select header is always sent together with the first negotiation message.

2. Enable 0-RTT: If the dialer only supports a single protocol, it can send protocol data (e.g. the actual application request) together with the multistream-select header and protocol proposal. Similarly, if the listener supports a proposed protocol, it can send protocol data (e.g. the actual application response) together with the multistream-select header and protocol confirmation.

3. In general, the dialer "settles on" an expected protocol as soon as it runs out of alternatives. Furthermore, both dialer and listener do not immediately flush the final protocol confirmation, allowing it to be sent together with application protocol data. Attempts to read from an incompletely negotiated I/O stream implicitly flushes any pending data.

4. A clean / graceful shutdown of an I/O stream always tries to complete protocol negotiation, so even if a request does not expect a response, as long as the I/O stream is shut down cleanly, an error will occur if the remote did not support the expected protocol after all.

5. If desired in specific cases, it is possible to explicitly wait for full completion of the negotiation before sending or receiving data by waiting on the future returned by Negotiated::complete().

Overall, the hope is that this especially improves the situation for the substream-per-request scenario.

The public API of multistream-select changed slightly, requiring both AsyncRead and AsyncWrite bounds for async reading and writing due to the implicit buffering and "lazy" negotiation. The error types have also been changed, but they were not previously fully exported anyway.

This implementation comes with some general refactoring and simplifications and some more tests, e.g. there was an edge case relating to a possible ambiguity when parsing multistream-select protocol messages.

Examples

0-RTT Negotiation

Logs of a 0-RTT negotiation; The dialer supports a single protocol also supported by the listener:

[2019-07-26T09:06:24Z DEBUG multistream_select::dialer_select] Dialer: Proposed protocol: /proto2
[2019-07-26T09:06:24Z DEBUG multistream_select::dialer_select] Dialer: Expecting proposed protocol: /proto2
[2019-07-26T09:06:24Z DEBUG multistream_select::listener_select] Listener: confirming protocol: /proto2
[2019-07-26T09:06:24Z DEBUG multistream_select::listener_select] Listener: sent confirmed protocol: /proto2
[2019-07-26T09:06:24Z DEBUG multistream_select::negotiated] Negotiated: Received confirmation for protocol: /proto2

A corresponding packet exchange excerpt (Wireshark) of 0-RTT negotiation where ping and pong are "application protocol data" and e:

> /multistream/1.0.0
> /proto2
> ping
< /multistream/1.0.0
< /proto2
< pong

Other Examples

Logs from a 1-RTT negotiation; The dialer supports two protocols, the second of which is supported by the listener.

[2019-07-26T09:15:01Z DEBUG multistream_select::dialer_select] Dialer: Proposed protocol: /proto3
[2019-07-26T09:15:01Z DEBUG multistream_select::listener_select] Listener: rejecting protocol: /proto3
[2019-07-26T09:15:01Z DEBUG multistream_select::dialer_select] Dialer: Received rejection of protocol: /proto3
[2019-07-26T09:15:01Z DEBUG multistream_select::dialer_select] Dialer: Proposed protocol: /proto2
[2019-07-26T09:15:01Z DEBUG multistream_select::dialer_select] Dialer: Expecting proposed protocol: /proto2
[2019-07-26T09:15:01Z DEBUG multistream_select::listener_select] Listener: confirming protocol: /proto2
[2019-07-26T09:15:01Z DEBUG multistream_select::listener_select] Listener: sent confirmed protocol: /proto2
[2019-07-26T09:15:01Z DEBUG multistream_select::negotiated] Negotiated: Received confirmation for protocol: /proto2

Since protocol negotiation "nests", the following may also be observed as a single packet (again a simplified Wireshark excerpt):

/multistream/1.0.0
/yamux/1.0.0
...
/multistream/1.0.0
/ipfs/kad/1.0.0
...

Multistream-select 2.0

Given the status of the spec and the apparent lack of any implementations that I'm aware of, I didn't actually bother trying to implement it. Instead, these changes incorporate minimal compatibility with the currently proposed upgrade path by having the listener (aka responder) always answer to /multistream/2.0.0 with na. The implementation could be continued once the spec becomes "actionable".