Adaptive Compression

[markqvist]

This PR implements adaptive compression for all data transfers. Since most interactive terminal data is easily compressible, this greatly improves performance over slower links.

Even when using rnsh for transferring non-terminal datastreams, this PR should improve performance and bandwidth utilisation in most cases.

Scatter-plot of compressibility vs tries and time to success. Heat indicates chunk size.

A representative sampling of 50.000 varied data-streams shows that the implemented methodology achieves high bandwidth savings within reasonable amounts of time (~3ms) for a significant amount of the stream chunks tested.

Almost all significant savings are found within 4 tries, and therefore the implementation has been capped at that amount of tries before data is sent uncompressed.

Normalised distribution of tries required for successful chunk compression

Since these changes rely on some under-the-hood additions to StreamDataMessage introduced in RNS version 0.5.9 (as of yet unreleased), I have bumped the required dependency version as well.

Please note that I have taken the liberty to include to log statement removals in this PR as well. I propose removing them, since they could inadverdently lead to users logging session I/O (think keystrokes) to a potential on-disk logfile, and at this point I believe it's better to not be able to accidentially to this. Feel free to revert those if I misunderstood something there!

Due to improvements in RNS in the upcoming release, it also seems that some of the strange behaviour and timeouts on packet loss, that did not recover, is now fixed.

[acehoss]

Looks great! I agree with dropping the log statements.

Once RNS 0.5.9 releases I'll rerun the CI build and merge this (assuming the CI succeeds).

[markqvist]

All good! I'll put a comment here when it's out.

[markqvist]

Reticulum version 0.5.9 has been posted, and the CI checks should go through now, but it seems a CVE for gitpython is blocking it now.

I'm not quite sure why that is, since I don't see the gitpython package used in rnsh anywhere. This may just be a problem with the CI tasks, and not rnsh itself.