feat: auto-tune (dynamic) stream receive window

[mxinden]

Motivation

Yamux provides flow control on the stream level, preventing a fast sender from overwhelming a slow receiver, in other words for the receiver to exerts backpressure to the sender. The receiver does so through communicating a flow control window in-band through WindowUpdate messages, basically granting the sender send-credit on a continuous basis.

The big question here is: how large should the window be. In other words, how much data should the sender be able to send, before waiting on another WindowUpdate from the sender. Choosing the window size too small results in the sender not being able to make use of the entire bandwidth. Choosing the window size too large results in a high memory footprint in the form of a buffer on the receiver side with the size of the window, and delays the backpressure signal from the receiver to the sender.

Status quo

Today the Rust Yamux window size is fixed. The default value is 256 KB. Assuming a connection round-trip-time of 60ms, this allows for up to 33 Mbit/s throughput per stream (256*1024/0,06*8=33.33). Obviously this is not ideal when running on e.g. a cloud instance with up to 10Gbit/s.

One can bump the default window size, to e.g. 16 MB. But then again, this might be too large for some deployments (low latency and/or low throughput) or too small for others (high latency and/or high throughput).

See #162 for a larger discussion on the various options.

Single stream throughput on a 60ms TCP connection with up to 5 Gbit/s machine bandwidth.

https://observablehq.com/@libp2p-workspace/performance-dashboard?branch=9a4c96952368e6432fe51fd12008d31ca30f63ea#branch

Solution

The solution is to auto-tune the stream receive window, making it dynamic based on the round-trip time and the estimated bandwidth, in other words aiming for the bandwidth-delay-product. This is nothing novel. In fact the Linux kernel auto-tunes the TCP receive buffer, QUIC implementations do it and the Go Yamux implementation does it.

Implementation

This pull request does the following:

• Send Yamux' Pings on an interval to measure the connection round-trip-time.
• Dynamically grow the stream receive window based on the round-trip-time and the estimated bandwidth. For a detailed analysis of the used algorithm see here and here. Long story short, we do what QUIC does.

Result

The results show that this pull request is a game-changer for any Yamux deployments. While low-resourced deployments maintain the benefit of small buffers, high resource deployments eventually end-up with a window of roughly the bandwidth-delay-product (ideal).

Same setup. Single stream throughput on a 60ms TCP connection with up to 5 Gbit/s machine bandwidth.

https://observablehq.com/@libp2p-workspace/performance-dashboard?branch=9a4c96952368e6432fe51fd12008d31ca30f63ea#branch

(As one can see, there is still room for improvement, i.e. 1.3 Gbit/s is not 5 Gbit/s. Though that might be due to some other parts of our stack. More to come in the future.)

---

Fixes #162.

Preliminary performance results: libp2p/test-plans#332

[thomaseizinger]

Did a preliminary review, results look great, well done!

I can see that you've spent a fair bit of effort to make the implementation of the algorithm easy to understand. I think it might be worth to improve on that further by decoupling it from the Stream struct. We should be able to implement the algorithm as a pure function and simply call it from Stream by passing in the required state. That should also allow you to test it more easily using quickcheck.

[thomaseizinger]

Does need a changelog entry!

[thomaseizinger]

Great work @mxinden!

[mxinden]

Tagged and published.

Thank you for your help @thomaseizinger!

[mxinden]

Using the libp2p perf tests, this change improves rust-libp2p's TCP/TLS/Yamux single stream performance from ~33 Mbit/s to 1.3 Gbit/s. See pull request description above for details.

I expect this change to have a large throughput performance impact on various live networks as well. Thus tagging folks here for visibility.

For those using yamux through libp2p, see libp2p/rust-libp2p#4970 integrating this change into rust-libp2p master, soon to be released as part of the libp2p v0.53 family.

Please reach out in case you have any questions.

• Sigma Prime: @jxs @AgeManning @divagant-martian
• Polkadot SDK: @altonen @bkchr @alindima
• Subspace: @nazar-pc @shamil-gadelshin
• Stormshield: @stormshield-pj50 @stormshield-frb
• Actyx: @rkuhn
• EigerCo: @oblique
• ... please tag others in case I missed someone.

[diegomrsantos]

@mxinden great work. Is there a plan to update the Yamux specification to cover this?

[mxinden]

Just to avoid confusion, none of this pull request requires changes at the protocol level. In other words, Rust Yamux follows/ed the libp2p Yamux specification before and after. Round trip time is measured via Yamux's PING message. Window updates are granted via Yamux's WINDOW_UPDATE message.

I am not planning to update the specification any time soon. Though I am happy to review a pull request. Want to give it a shot @diegomrsantos? Suggestions to add:

• Regularly measuring round-trip time via Yamux's PING messages.
• Granting and increasing window based on QUIC's flow-control algorithm.

[thomaseizinger]

Slightly offtopic but I think this is actually a great showcase on what good protocol & API design requires: Specification of messages and their associated behaviour. Simply by specifying the ping & window update frames, it is possible to build quite elaborate algoritms for improving performance, whilst being entirely backwards compatible.

I don't mind if this get mentioned in the spec as a "further resource" or something like that but it should be very clear that it is entirely an implementation detail when window updates are sent and how much credit is given to the remote.