You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
One of the lotus-soup tests is exercising FIL deals with Lotus, by creating a network with a few clients and miners, and clients concurrently submitting deals to miners. Since deals are created concurrently within the span of a single block, sometimes the funds are not available on chain and the client sends immediately a proposal to the miner, which results in a rejected deal.
Ultimately the client fails with the following error:
ERROR storagemarket_impl clientstates/client_states.go:289 deal bafyreicnknok2ndpsaqv2f2il545gm5jarckc65evthxlpeb44kwclwde4 failed: unexpected deal status while waiting for data request: 11 (StorageDealFailing). Provider message: deal rejected: clientMarketBalance.Available too small
To Reproduce
Steps to reproduce the behavior:
Trigger a CI job on TaaS with testground run composition -f _compositions/deals-stress-concurrent-natural-k8s.toml from the filecoin-project/oni repo, lotus-soup directory.
See error on https://ci.testground.ipfs.team/
(This test runs on a regular basis, so it is possible to see a past failure as well).
(TaaS is not yet available to everyone, so talk to me if you need access).
Expected behavior
Clients should be able to send a few deals concurrently to a given miner successfully.
Version (run lotus version):
v0.10.0
Additional context
Markets code calls node.EnsureFunds() 6 times, each time increasing by 640297000 (the cost of one of the deals). This means it calls EnsureFunds(640297000), EnsureFunds(640297000 * 2), etc.
The node processes one of these calls out of order. This is ok, because the amount requested will still cover all 6 deals. What happens in the logs is it calls in this order: EnsureFunds(640297000), EnsureFunds(640297000 * 3), EnsureFunds(640297000 * 2), ...
So the 3rd call sees that there is enough funds for its requirement, and exits without sending an AddBalance message. This should generally be ok, because the we are still ensuring that there is enough to cover outstanding deals.
Closing this ticket as the Legacy Lotus/Lotus-Miner Markets sub-system reached EOL at the end of the 31st January 2023, and is being completely removed from the code here:
Describe the bug
One of the
lotus-soup
tests is exercising FIL deals with Lotus, by creating a network with a few clients and miners, and clients concurrently submitting deals to miners. Since deals are created concurrently within the span of a single block, sometimes the funds are not available on chain and the client sends immediately a proposal to the miner, which results in a rejected deal.Ultimately the client fails with the following error:
To Reproduce
Steps to reproduce the behavior:
testground run composition -f _compositions/deals-stress-concurrent-natural-k8s.toml
from thefilecoin-project/oni
repo,lotus-soup
directory.(This test runs on a regular basis, so it is possible to see a past failure as well).
(TaaS is not yet available to everyone, so talk to me if you need access).
Expected behavior
Clients should be able to send a few deals concurrently to a given miner successfully.
Version (run
lotus version
):v0.10.0
Additional context
node.EnsureFunds()
6 times, each time increasing by 640297000 (the cost of one of the deals). This means it calls EnsureFunds(640297000), EnsureFunds(640297000 * 2), etc.EnsureFunds(640297000)
,EnsureFunds(640297000 * 3)
,EnsureFunds(640297000 * 2)
, ...AddBalance
message. This should generally be ok, because the we are still ensuring that there is enough to cover outstanding deals.AddBalance
message was sent, funds are available, and we continue deal negotiation, immediately sending the proposal to the miner (https://github.com/filecoin-project/go-fil-markets/blob/master/storagemarket/impl/clientstates/client_states.go#L70-L73)AddBalance
calls have not appeared on chain yet.cc @ingar @hannahhoward
The text was updated successfully, but these errors were encountered: