[Forknet] MVP of stateless validation upgradability test in CI

[posvyatokum]

This is a tracking issue for the April effort of using forknet to add transition to stateless validation into our CI.

Tasks

Preview Give feedback

1. Incorporate fork-network tool into mocknet creation flow
2. Create a regular job of running current setup for stateless validation upgradability test
3. Support StatelessNet/Forknet for Stateless Validation development #11056
   A-stateless-validation +1
   

[posvyatokum]

@marcelo-gonzalez When you have time, add tasks that you can think of to the tasklist in the description.

[posvyatokum]

@marcelo-gonzalez I see our basic improved flow like this:

Preparation:

Independently of the test and in advance of starting it, prepare a node image.
Node image is prepared via running init, amend-access-keys, and finalise in fork-network tool.

Test Setup (new part):

• Create a network of nodes from prepared images.
• Generate validator keys for all nodes
• Distribute validator keys
• Distribute list of validators and stakes to all nodes
• Run fork-network init and fork-network set-validators on all nodes. Don't run finalise, as this will allow us to quickly revert back to original state by running reset.
• Amend genesis config separately as a json (we can do it without any extra logic, just by running jq or something)

Test Setup (± old part):

• Generate, and distribute node keys
• Change configs (including boot_nodes)

Test (no changes):

• Start nodes
• Start traffic

I see us generating and distributing keys instead of using neard on machines to generate them for two reasons:

• I don't know how init command are going to play with home dir that gone through fork-network
• This flow is more natural in my mind (purely subjective)

I prepared this draft PR to show how I see keys management in forknet. Please, take a look #10956.