Consider QueryOffer Error & Status

[jsign]

Consider QueryOffer.Status before calling ClientRetrieve, and be aware of non-errored Retrieval.Query calls that have non-success .Status.

Continuation of #2040
cc @whyrusleeping

[jsign]

Despite it's the default value, preferred to be explicit about the case.

[mikeal]

This may be a bigger/broader issue than just this PR but it’s relevant so I’m going to bring it up.

The PayloadCID is not secure and whenever we use it without the PieceCID there are possible attacks.

The PayloadCID is the hash of a root block inside a CAR file.

It’s actually not difficult at all to take that CAR file, store only the root block, and do a new deal on that CAR file.

The only place we have assurances that the rest of the data for the graph is actually stored in the CAR file is in the default lotus dealflow (client and miner agree on CommP), so it’s possible to do deals for CAR files that don’t have all of the data using their own deal flow (the offline deal flow would allow this).

What happens here when a miner has two storage deals for the same PayloadCID?

[hannahhoward]

@mikeal offline deals are specified with a PieceCID and the car file that is received is validated to match the PieceCID before it's accepted for the deal.

[hannahhoward]

also: the reason retrieval uses payload CIDs is you're not paying for "the piece" you're paying for data -- and in fact, what you care about is that you can verify that data incrementally, because you pay for it incrementally. If the miner does not serve the whole payload (or the payload + requested selector) you don't pay for it.

You can however, also specify the pieceCID, so you're saying "I want this payload, served specifically from this piece".

However, in general, retrieval is a proof is in the pudding protocol -- the miner can say they have anything, and the intent is that retrieval providers may not be miners at all -- they may just be people holding unsealed data. What you pay for is what they send you. And the good thing about a payload is we can use graphsync & selectors to know everything we're getting is part of the payload as we get it.

[hannahhoward]

But long and short, the required parameter is payload CID -- otherwise you have no way to know till the whole deal is transferred that you've gotten what you asked for.

[hannahhoward]

Important Follow-up: I see now that we never exposed the parameter for PieceCID on query. I'm sorry about that. I will elevated that ticket.

[mikeal]

I may need to hold off on this until we can have a deeper call on how payment works in the retrieval market, but here’s a few questions I have (feel free to wait to answer them until we have time for a larger brain dump).

• Since this transaction is trustless:
  • How does the client cancel payment if it only receives partial data?
  • How do we trust that the client isn’t lying in order to avoid payment?
• If we don’t use PieceCID along with PayloadCID for retrievals can’t I craft a denial of service attack on a piece of content by creating new storage deals with that miner for CAR files of incomplete data w/ the targeted PayloadCID?