Use PiecePicker for arbitrary things #536
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
TODO:
|
The engine now uses the exact same logic for 'picking' bittorrent v2 hashes, fetching the metadata and also regular piece/block downloading. This makes failure modes much easier to reason about. Additionally, it's much easier to ensure that the data is reliably fetched from other peers as the fault tolerance of the piece picking engine will ensure things are received, or re-requested from another peer.
alanmcgovern
force-pushed
the
pick-anything-using
branch
from
cfa86e4 to
8f47fea
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PiecePicker has been simplified to have the concept of having a
Piece, where each piece has 1 or moreSegments.The invariant is that once you download all the segments in 1 piece, the piece can be validated, allowing the data to be requested again if validation fails, or if it succeeds then the piece can be marked as such so it isn't requested again.
In concrete terms, there is no change to regular piece picking. However,
Fetching torrent metadata
The chunks are not independently verifiable so this data is downloaded as a single piece with N segments. Once all segments are received the data is hashed.
Fetching BitTorrent v2 hashes
The chunks are independently verifiable, so this data is represented as a series of pieces, each of which has 1 block. As soon as any block is received the data is verified.
This makes it much easier to double-request pieces, or to time out requests which aren't fulfilled due to a bad client improperly responding.