3.13 Testplan

[ramezsaeed]

Introduction

Test plan for v 3.13.x release on TFgrid

• development project: https://github.com/orgs/threefoldtech/projects/199
• Known Issues: Known issues for 3.13.x release #1490

Schedule

• 3.13.x expected to be on QAnet on: ----
• All tests and verification should be done on QAnet within one week maximum.
• After deploying on Testnet will keep it for one week before go to Mainnet in order to get tested by the community.

Scope

Test the main features in 3.13.x release:

• New playground (dashboard)
• Farmerbot
• Mass Deployment (TF Robot)
• Mycelium

Test Reports

• Testlodge Run:
  • QAnet:
    • QAnet 3.13 TestRun V1
  • Testnet:
    • TESTnet 3.13-rc2 TestRun V1
  • Mainnet:
    • Mainnet 3.13 TestRun

Roles and Responsibilities

1. verification are done on Devnet (team members verify/ Ramez can close)
2. release on QAnet done by dev team approval.
3. QA team starts new test run iteration on Qanet.
4. BAM(Bug Assessment Meeting) is needed to approve the release and address the reported issues.
5. once approved the release go to Testnet.
6. QA team starts new test run iteration on Testnet.
7. BAM is needed to approve the release and address the reported issues.
8. once approved release kept on Testnet for two weeks before go to Mainnet.
9. once release on Mainnet QA team starts new test run iteration on Mainnet.

This test plan is maintained by QA team:

@ramezsaeed
@khaledyoussef24
@A-Harby

Main Features Testing and Validation:

• Migrate Dashboard to vue3 tfgrid-sdk-ts#468

  • Testlodge coverage: https://app.testlodge.com/a/26076/projects/40893/suites/234374
  • Related issues:
    • Automating the new playground and dashboard tests tfgrid-sdk-ts#1285
    • improve user experience TFGrid #1466

• rewrite the farmerbot in go tfgrid-sdk-go#484

  • Testlodge coverage: https://app.testlodge.com/a/26076/projects/40893/suites/221831
  • Related issues:
    • make sure farmerbot is part of the grid verifications #1464

• tfrobot: easy mass deployer for VMs #1504

  • Testlodge coverage: https://app.testlodge.com/a/26076/projects/40893/suites/236972
  • Related issues:
    • Add mass deployer pkg tfgrid-sdk-go#604
    • Investigate the performance and time taken by the mass_deployments workflow tfgrid-sdk-ts#1352
    • mass-deployer: testing checklist tfgrid-sdk-go#710

• ZOS mycelium v1 #1480

  • Testlodge coverage: https://app.testlodge.com/a/26076/projects/40893/suites/235270
  • Related issues:
    • integrate mycelium in zos zos#2073
    • to be able to run mycelium next to yggdrasil please find an alternative ipv6 subnet mycelium#33
    • story: mycelium v 0.9  #1465
    • Support mycelium on grid client deployments tfgrid-sdk-ts#2085

Non Release Blocking Stories:

• KYC requirement for customers + strong verification #1450
  • please check carefully after this got deployed
  • add new test cases in Testlodge
• improve the gpu indexing with the change in number of nodes tfgrid-sdk-go#322
  • Test cases here: https://app.testlodge.com/a/26076/projects/40893/suites/228929?expand_section=423253#suite_section_423253

Out of QA Team Scope

• ZOS API v4 #1468
• threefold cloud website #1459
• formalize TF Validators #1451
• gather more info from 3nodes #1453
• tfchain activation service maintainability #1441
• BSC bridge multisig improvements #1424
• fix publishing tools #1463
• 3bot reorg: part2 #1482

[ramezsaeed]

work in progress (WIP)

• verfication on devnet for newplaygroun
• newplayground testsuite on testlodge
• fix automation nightly runs
• refactor old dashboard automation tests to run on the newplayground.

[ramezsaeed]

As per discussion in BAM 18 Dec 2023;

• Wait for Farmerbot testing
• these stories will be moved to 3.14

• story: mycelium v 0.9  #1465
• FungiStore: version 1 #1467

[ramezsaeed]

Work In Progress (WIP):

• Verification on 3.13 tfgrid-sdk-ts
• Farmer-Bot ready for Testing (But can't start in it yet due to all resources focus on 3.13 UI release)
• Automation for new dashboard update Homepage, Profile Manager, Twin and Transfer and Bridge tests tfgrid-sdk-ts#1667

[ramezsaeed]

Work In Progress (WIP):

• Verification on 3.13 tfgrid-sdk-ts (Done)
• Automation for new dashboard (Done)
• more UI changes had been requested by KDS will be handled

[ramezsaeed]

Work In Progress (WIP):

• Improve all dashboard changes (Done) improve user experience TFGrid #1466
• Mass deployment test coverage (Done) mass-deployer: testing checklist tfgrid-sdk-go#710
• FarmerBot (WIP) Create Farmerbot testsuite on testlodge tfgrid-sdk-go#787
• Solutions (WIP) Test Automation for All available solutions on the new dashboard tfgrid-sdk-ts#1841

[xmonader]

Already released

[ramezsaeed]

[khaledyoussef24]

Work done
farmerbot : no issues found only some commands in the docs are missing quotations

test run on main net for 3.13 is done

skipped tests mostly related to terraform automated tests