Combined EVA-UDP-caching RQC protocol

[ichorid]

Currently, when browsing Channels, RQC issues a request over a UDP packet, which is later handled by EVA which sends all the contents from the remote database every time, without regard to the previous communication. This results in the same transfers repeating again and again.
An average request is 200 Bytes of channel entry * 50 times = 10k bytes, which is about ~1 EVA default window size. The number of roundtrips is at least:

1. A->B: Request over UDP RQC
2. B->A: Response with "EVA push handshake" message
3. A->B: Response with "confirm EVA handshake" message
4. B->A: Pipeline send 10k bytes of response data
5. A->B: Confirm receive, possible repeat step 4.
   Which is at least 2 full round trips (probably more).

It is possible to reduce this to a single round trip by using caching via a set reconciliation library and combining the initial handshake with the request, QUIC style. So the exchange would typically look like:

1. A->B: Request over UDP RQC (sketch of A's answer to the same query + request + EVA handshake)
2. B->A: Send the missing response data
3. A->B: Confirm receive
   Which reduces the number of roundtrips by at least 1 (for the handshake), and 0-X (because of the caching achieved by using sketches).

It is possible to employ a fast symmetric sketch library, such as minisketch for this purpose. A typical RQC request while browsing Channels asks for 50-100 entries. An entry's ID can be coerced to 32 bit number, which will result in a maximal set difference of 100 entries of 32 bits. For minisketch a sketch of 100 entries of 32 bits will take 3200 bits, which is just 400 bytes. This leaves more than 600 bytes for the rest of the query and EVA handshake.

For even simpler implementation, we can even omit using a sketching library, since the same 400 bytes will still fit into a UDP packet verbatim.

Such an optimisation should increase the perceived Channels browsing speed 2-3 times.

[synctext]

This leaves more than 600 bytes for the rest of the query and EVA handshake.

Have you tried "Dogfooding Tribler 7.10"?

Impressive thinking, but the whole metadata part of Tribler 7.10 is not good enough yet for this level tuning. Plus too many open bugs in our issue tracker. My simple usability tests of Channels indicate that its actually in too bad shape for me to put an expert on it: phd student in usability and/or user experience. Tribler should be fun to use. That's not about "perceived Channels browsing speed 2-3 times".

[ichorid]

Have you tried "Dogfooding Tribler 7.10"?

Well, I use it daily. Indeed, Channels Search should be the focus now. That's why I added this issue to the Backlog milestone since the beginning 😉

[drew2a]

It is not relevant anymore:

• Channels Retirement #7669