approval-voting improvement: include all tranche0 assignments in one certificate

[sandreim]

fixes paritytech/polkadot-sdk#729

This PR will upgrade the network protocol to version 2 -> VStaging which will later be renamed to V2. This version introduces a new kind of assignment certificate that will be used for tranche0 assignments. Instead of issuing/importing one tranche0 assignment per candidate, there will be just one certificate per relay chain block per validator. However, we will not be sending out the new assignment certificates, yet. So everything should work exactly as before. Once the majority of the validators have been upgraded to the new protocol version we will enable the new certificates (starting at a specific relay chain block) with a new client update.

There are still a few things that need to be done:

• Use bitfield instead of Vec<CandidateIndex>: approval-voting: implement some TODOs and remove v2 assignment core_indices and samples #6802
• Fix existing approval-distribution and approval-voting tests
• Fix bitfield-distribution and statement-distribution tests
• Fix network bridge tests
• Implement todos in the code
• Add tests to cover new code
• Update metrics
• Remove the approval distribution aggression levels: TBD PR
• Parachains DB migration (wipe column): included in approval-voting: implement some TODOs and remove v2 assignment core_indices and samples #6802
• Test network protocol upgrade on Versi
• Versi Load test
• Add Zombienet test
• Documentation updates
• Fix for sending DistributeAssignment for each candidate claimed by a v2 assignment (warning: Importing locally an already known assignment)
• Fix AcceptedDuplicate
• Fix DB migration so that we can still keep old data.

[Polkadot-Forum]

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/dynamic-backing-groups-preparing-cores-for-agile-scheduling/3629/3

[ordian]

Going to give it an approval to signal readiness for an audit.

Just to be clear, before merging this, I expect to see the following addressed:

• merge conflicts resolved after async backing is merged
• Versioned::VStaging is stabilized into Versioned::V2
• Another Versi burn-in after these changes
• audit (when v2 is actived)

[ordian]

Does this need to addressed before merge? If not, don't forget to create an issue :)

[sandreim]

Going to give it an approval to signal readiness for an audit.

Thanks !

Just to be clear, before merging this, I expect to see the following addressed:

• merge conflicts resolved after async backing is merged

Yup, this is the next step.

• Versioned::VStaging is stabilized into Versioned::V2

My plan is to merge in master and piggy back on the network upgrade for async backing. Currently master code has the same naming with same semantics. But yeah, if we remove the feature flag until I merge, then it will have to be V3.

• Another Versi burn-in after these changes

Yes, this is in progress implicitly as we burn it along in #7554 , but I will do a separate test before the merge.

• audit (when v2 is actived)

I am assuming we want to do the audit before the merge, then we are more flexible to when we enable the new code.

[paritytech-cicd-pr]

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: test-linux-stable
Logs: https://gitlab.parity.io/parity/mirrors/polkadot/-/jobs/3424355