State migration consistence test when upgrading

[steven004]

To answer the question whether each implementation is functioning the same way in the next network version, we had review, butterfly network and calibration network testing, aside from all of these, could we do more to line up directly on the main net?

Here is a proposal: whenever there is a new network version with state migration, we do upgrading simulation directly on main net at a particular height. Every implementation sync the main net, upgrades at the same height, and report the new state root right after the states migrated. Of course, all the roots provided by different implementations must be identical.

Could we put this into our upgrading process?

[LexLuthr]

@steven004 Just to clarify, venus and lotus both are running and new network version is coming at epoch x. So when both nodes sync to epoch x, we match the stateroot of x(or parentstateroot of x+1) on both nodes and make sure it is the same. If there are more implementations then we do the same with them as well. Is this the suggested test? Where do you propose should this test go in an upgrade process and what advantages would it provide?

[steven004]

Yes. It is the suggested test.

I did not mean the actual upgrading process by a user. Maybe I should say the network upgrade testing process. I would think this can be done before publishing a release for each implementation.

Clearly it will tell the implementation maintainers that your states transaction implementation is correct. One may say that you can do it in calibration network. Yes, you can do it, but main net contains lots of things the calibration network does not. The calibration network just can do simulation at some level and much smaller. If we would do it in real world (the main net) without much cost, I would like to get more confidence from it.

[LexLuthr]

@steven004 Thanks for clarifying everything. As jennijuju has already said, this is a good idea and it will be added to the list.

[jennijuju]

@steven004 This sounds like a reasonable check to add. I'll add it to our ntwk v16 upgrade readiness checklist.

[arajasek]

Yes, definitely! Thanks for the idea @steven004, and it's not too late to do this for v15! If you can suggest a tipset, we can run the migration in Lotus and report our state root!

[hunjixin]

is 1582155 okay?, we plan to test migration time cost at this epoch.

[hunjixin]

migration result:

fork/fork.go:480        COMPLETED migration     {"height": "1582155", "from": "bafy2bzacebblfqy5k42rjelqcmqlfujnk4led2cumjh4tjaywrao7vyzlss4c", "to": "bafy2bzacebgtq7it2vl2m2ps34ydowwmwiyyybejwih6yfgkf7ym6bkzlbwck", "duration": 10.581965997}

the tipset root of 1582156

state mis-match computed state (bafy2bzaceb7tueg7fxnsd7ddfq3q2vljlqqmylyujmcbcp2tttqm6htqhguiw != bafy2bzaceblyrd3er5qxjze4azimgkoiziitlss37mqce63mq6wn3hyczdeju),blocks state root does not match computed result:

[kaitlin-beegle]

This is a great idea, @steven004!!

In order to make this more visible, I started a new discussion in the TPM repo and suggested a tipset for testing. I'll flag for each implementation team outside of Github as well.

CC for migration checklist: @jennijuju

[LexLuthr]

I am archiving this discussion as it has been moved to a new thread.