QA Epic Task list to pull from

[kravciak]

• Clean up playwright.yml github action according to vs code recommendations

• ClusterAdmissionPolicies vs AdmissionPolicies
  kwctl scaffold --type?

• add html report to github pages
  Add html report accessible from github pages to see test results
  Add link to report into job summary - GITHUB_STEP_SUMMARY

• fix upgrade tests

  • perform the upgrade-from-version-under-test to version-under-test (tests upgradability of version-under-test so we can ship fixes in the future that can be applied)
  • recreate the cluster with the current released version, and perform the upgrade to the version-under-test (tests that people can upgrade to version-under-test)

• Parametrize Kubewarden installation
  Add ability to choose kubewarden source (released | rc | source) on manual run in github action
  Allow to specify version: head | rc | released | tag#1.0.4-rc2 | pr#xxx | v#1.0.3

---

• github action installs cluster but Makefile prints diferent versions
  Tests in helm-charts actions for example

• check that our policies have icons, look for cosign / official icons

• test matrix:

  • rancher released|rc
  • kubewarden released|rc

• ClusterAdmissionPolicies vs AdmissionPolicies
  can I test all of them as as CAP?
  what happens when I swap them?

• policy tests: delete policy in post-hook (after failure)
  When policy fails to start it's not deleted and all follow-up policies fail too as a consequence. It can be prevented to move deletion to post-hook.

• print versions of installed components
  Currently it's not obvious from test report which components were used in tests. Add info to GITHUB_STEP_SUMMARY.

• UI testsuite improvements

  • Rewrite policies test into POM
  • Add table handler class

• Display Helm Charts rc versions based on user preferences
  User -> Preferences -> Show released only | Include prerelease versions

• Parametrize Rancher installation
  Add ability to choose rancher version (rc|released) on manual run in github action
  Allow to specify version v2.7.1 | v2.7.2-rc7,

• Parametrize Kubernetes version
  Test on kubernetes 1.25 - ranacher 2.7.2 brings 1.25, move to this version and add option for manual test run to use older one. Maybe add both to test matrix, but as discussed on the call it should not have much effect on functionality.

• Use locator.or instead of Promise.race
  https://playwright.dev/docs/other-locators#combining-two-alternative-locators

• delete package tests/psp-user-group
  https://github.com/orgs/kubewarden/packages/container/package/tests/psp-user-group

• rewrite to POM
  Reorganize code to follow playwright page-object-model

• replace :near selectors
  Replace with filtered locators, using :near is not reliable

• add helper functions / locators to playwright
  label.checkbox-container -> check | uncheck
  move merge yaml to global helpers

• E2E test is failing
  started to fail after adding private registry and context aware?

• Fix failing e2e job on helm charts
  E2E tests timeout when installing kubewarden-controller chart kubewarden/kubewarden-end-to-end-tests#57

• export test report from all playwright tests
  Playwright produces test report, but directory is overwritten by follow-up runs (installation -> policies -> other tests)

• use yarn build * server instead of dev
  Dev uses quite a lot resources and github runner has only 8GB RAM - this change might make it OK to use runner again
  Try to move back to github runner

• rewrite to typescript
  Move from javascript to typescript

• test policies from authenticated registry
  fix: allow Policy Server to consume policies from authenticated registries kubewarden/kubewarden-controller#421