Add Guide For Network Scale Testing (Versi)

[BradleyOlson64]

Add a body of documentation covering network scale testing such as using Grafana with Versi and perhaps Zombienet/CI. This would be quite useful for new hires and devs who are rusty with it. Should be added as "Debugging Guide" mdBook in the Polkadot repo.

First draft topics to cover:

• Prerequisites
  • Parity VPN
  • Grafana dashboard editor permissions
  • GitLab SSH key setup
  • Clone parity-testnet repo from parity GitLab
• Getting to Know Grafana
  • Provide a variety of example log queries with different properties EX: Query all successful approvals
  • Walk through constructing a log query, including pictures of Grafana screen
  • Provide links to example dashboards with different properties
  • Guide user through making their first dashboard with variables and panels
• Metrics and logs in the code
  • Provide links to PRs adding logs and adding metrics (EX: Testing Reversion Speed on Dispute Concluded Against polkadot#6880, Adding Dispute Participation Metrics polkadot#6838)
• Getting data from Zombienet/CI
  • Check CI test logs for folder to target
• Modifying Versi Nodes for Test
  • If new logs or metrics then get PR reviewed and submitted
  • Get docker image
  • Add to validators helmfile and change other validator group configs if necessary
  • Make branch of parity-testnet repo, push to remote on GitLab, ASK WHETHER VERSI IS AVAILABLE first in Parachains Core element channel, then merge the branch without reviews
  • View with query as nodes spin up with new node image (~10 minutes)
  • Adding and configuring malus nodes

[mrcnski]

Awesome idea and would be very helpful! I guess you're already familiar with this wiki but I think it covers some of these topics, especially the prereqs, so you can maybe just link to it for those. There is also the Infrastructure Overview in case it's relevant at all.

[BradleyOlson64]

Reseting Versi: https://github.com/paritytech/devops/wiki/Resetting-Testnets#reset-versi-fast-way

[BradleyOlson64]

Testnet wiki to reference: https://github.com/paritytech/devops/wiki/Kubernetes-Experimental-Testnets-(Versi)