Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage Client Statemachine Refactor #136

Merged
merged 5 commits into from
Mar 6, 2020

Conversation

hannahhoward
Copy link
Collaborator

Goals

Make the storage client

  • better commented
  • easier to navigate and look at code for
  • more well tested

Implementation

  • Setup a client state machine using all the FSM module in go-statemachine
  • Test all state handlers with unit tests
  • Define events for everything that happens in the storage client module
  • Add additional infrastructure to support more testing
  • Setup networking mocks for streams

Refactor the storage client to use finite state machines - also add some more fine grain state to
the beginning of deals on the client
Setup testing infrastructure for testsnodes and add several unit tests for different client states
@hannahhoward hannahhoward force-pushed the feat/storage-market-state-machine-testing branch from b24b172 to bcad340 Compare March 6, 2020 01:07
Copy link
Contributor

@ingar ingar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, seems cleaner overall, thanks for the data xfer request validation stuff also.

deal := &storagemarket.ClientDeal{
ProposalCid: proposalNd.Cid(),
ClientDealProposal: *clientDealProposal,
State: storagemarket.StorageDealUnknown,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that these states predate this PR, but I find the use of StorageDealUnknown too overloaded. Maybe in another PR we can separate this out, since it currently means StorageDealInitial, StorageDealWaitingForSomeEvent, StorageDealUnknown, etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree will fix in future PR

c.handle(ctx, deal, c.sealing, storagemarket.StorageDealNoUpdate)
// TODO: StorageDealActive -> watch for faults, expiration, etc.
}
func (c *Client) Run(ctx context.Context) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this can boot up in-progress deals when restarting the node?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, "in the future" :)


// ClientEvents are the events that can happen in a storage client
var ClientEvents = fsm.Events{
fsm.Event(storagemarket.ClientEventOpen).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite remember why we send an event like this. Is it because we need an arbitrary event for the FSM to start running? Doesn't Begin() do that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no Begin() does not do that -- it sets an initial state, but there is no implied initial event

// ClientEvents are the events that can happen in a storage client
var ClientEvents = fsm.Events{
fsm.Event(storagemarket.ClientEventOpen).
From(storagemarket.StorageDealUnknown).To(nil),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be .ToNoChange()?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right. #whenYouForGetYourOwnDSL

return nil
}),
fsm.Event(storagemarket.ClientEventDealProposed).
From(storagemarket.StorageDealFundsEnsured).To(storagemarket.StorageDealValidating),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, I wonder what the race condition recovery looks like here with multiple simultaneous deals..

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well the upside is the provider validates it too... so maybe they'll catch it? But you are right that the logic ought to be ensure funds for all deals in progress... dunno what that looks like.

return ctx.Trigger(storagemarket.ClientEventDealProposed)
}

// VerifyDealResponse reads and verifies the response from the provider to the proposed deal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should read, "VerifyDealResponse waits, waits, and waits, and waits some more, and then reads and verifies..." :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omg... I am very ready to send that intent to accept a lot earlier and switch up the data transfer initiation... just gotta finish this here refactor first.

@codecov-io
Copy link

Codecov Report

Merging #136 into master will increase coverage by 19.09%.
The diff coverage is 91.45%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #136       +/-   ##
===========================================
+ Coverage   37.13%   56.21%   +19.09%     
===========================================
  Files          38       31        -7     
  Lines        2869     2055      -814     
===========================================
+ Hits         1065     1155       +90     
+ Misses       1506      614      -892     
+ Partials      298      286       -12
Impacted Files Coverage Δ
storagemarket/impl/requestvalidation/types.go 0% <ø> (ø)
storagemarket/types.go 100% <ø> (ø) ⬆️
...agemarket/impl/requestvalidation/types_cbor_gen.go 0% <ø> (ø)
storagemarket/impl/clientstates/client_states.go 100% <100%> (ø)
storagemarket/impl/clientstates/client_fsm.go 100% <100%> (ø)
storagemarket/types_cbor_gen.go 17.5% <30.77%> (+1.03%) ⬆️
...pl/requestvalidation/provider_request_validator.go 90.48% <90.48%> (ø)
...impl/requestvalidation/client_request_validator.go 90.91% <90.91%> (ø)
storagemarket/impl/provider_cbor_gen.go
storagemarket/impl/provider_asks.go
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed9da26...1c7674b. Read the comment docs.

@hannahhoward hannahhoward merged commit ae0c823 into master Mar 6, 2020
@hannahhoward hannahhoward deleted the feat/storage-market-state-machine-testing branch April 30, 2020 21:33
@dirkmc dirkmc mentioned this pull request Jan 19, 2021
@dirkmc dirkmc mentioned this pull request Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants