Introduce stateless offline dealflow, bypassing the FSM/deallists #5961
Conversation
This is aproposal for an additional flag --manual-stateless-deal and a corresponding API endpoint ClientStatelessDeal. This allows firing off an offline-style deal against a miner without keeping further track of it locally. Not keeping any local state introduces the limitation of requiring free storage deals, as there is nothing to tie the payment channel setup to. Rationale/need for this type of flow is the case of incredibly large sets of data nd deals, where the client and providers have prearranged payment ahead of time, and the client has a separate-from-lotus database of deal inventory. This way the client can use their lotus node merely as a network gateway, without running into any limitations currently present in both lotus as a whole and go-fil-markets in particular. Specific context for this work is filecoin-discover, where the requirement is to onboard ~ 12,000,000 individual deals against a pool of miners with whom the client has prearranged a relationship.
ribasushi
requested review from
Kubuxu,
magik6k and
whyrusleeping
as code owners
|
Are there any additional tests we should be adding? During w3dt project leads meeting, I saw @raulk post that we should have a testground test that stresses this path. |
|
Mmmmm this codepath shuts off pretty much all of client lotus, reducing everything to a single fire-and-forget libp2p message: there's nothing to stress. Stress testing would be required on the miner side, since we can't do the same "short circuit" there. But that's a way larger endeavor for which a lot of ignite-like domain knowledge is required. That said, a basic test can't hurt, provided there is interest in this functionality in core lotus the first place. |
|
@pooja : we missed talking about this during triage. It was a truncated session and Magik is also out this week and next. Can this wait until his return or does it need to get done sooner? I'm bummed that we have PRs open for so long (not good). The team is also being asked to make Lotus 1.10 (with FIP8/13 integration) the top priority. |
|
@BigLep rebased/updated for master / apiv1 |
@arajasek I have been using this on mainnet / in production since I opened the PR, so it has about a month of mileage now. |
That's all I need to hear :) |
( read diff under
-w)This is aproposal for an additional flag --manual-stateless-deal and a
corresponding API endpoint ClientStatelessDeal. This allows firing off
an offline-style deal against a miner without keeping further track of
it locally.
Not keeping any local state introduces the limitation of requiring free
storage deals, as there is nothing to tie the payment channel setup to.
Rationale/need for this type of flow is the case of incredibly large
sets of data nd deals, where the client and providers have prearranged
payment ahead of time, and the client has a separate-from-lotus database
of deal inventory. This way the client can use their lotus node merely
as a network gateway, without running into any limitations currently
present in both lotus as a whole and go-fil-markets in particular.
Specific context for this work is filecoin-discover, where the requirement
is to onboard ~ 12,000,000 individual deals against a pool of miners
with whom the client has prearranged a relationship.