rpm-ostree should have a multi-threaded, multi-piece, and multi-route download processs

[da2ce7]

on Fedora Linux 38.20230915.0 (Silverblue)

part of:

• Bad Experience on Unreliable Connections #2560

Problem:

rpm-ostree download process is slow under adverse conditions

Long Timeouts.

Slow Reconnections.

Single Routes.

User Experience:

Users with unstable connection have a very poor download experience, even to the extreme that the download process thrashes and will never complete.

Leading slow updates and a overall poor experience.

Expected Behavior:

Downloading should be resilient to adverse internet conditions. Using multiple connections downloading multiple pieces from multiple sources.

Available bandwidth should be dynamically calculated and bandwidth should be throttled not to congest the users connection.

Timeouts should be short and attempts should be staggered so that unstable conditions can be detected and averaged out.

Consider using robust transfer protocol like: rsync or Micro Transport Protocol or μTP.

related:

• rpm-ostree should be very respectful of user bandwidth #4599

Would you like to work on the issue?

For the moment I'm working on unrelated bittorrent infrastructure; I believe that downloading data in reliable and secure way is already a solved problem.