K6 Test Suite

[ivanape]

Feature Description

Objective: to be able to somehow define that a set of k6 test cases (or a set of k6 CRDs from the operator's point of view) are considered as a group and that start/stop operations apply on that set of k6 scripts.

In this way, we could define a load test scenario (or suite) that will consist of several k6 scripts. For example:

scenario/suite: black-friday
|------- test-case-navigate-products.js
|------- test-case-creating-carts.js
|------- test-case-checkout.js
|------- test-case-update-user-profile.js
|------- some-another-fancy-test.js

The operator should treat this set of tests as a unit and perform all start and stop operations together.

Suggested Solution (optional)

Option 1: Create a new CRD, (i.e. K6Stuite) with a custom controller to handle the test cases groups.

This new definition could be as simple as an array of k6 CRDs defining the group of test cases to be executed.

Perhaps this option is the most conservative since this new runtime will have a specific controller associated with it and would not share code. To the resources created by this controller, instead of creating the label "k6_cr", a label "k6_suite" could be added so that they are discarded by the existing controller "k6_controller".

This new controller would be responsible for creating the 3 necessary jobs (initializer, runner and starter) for the load test, taking into account the particularities of this execution model.

Option 2: Create the relationship of several k6 CRDs using labels/annotations and handle the groups with existing controllers.

With this approach, a new field "k6_suite_name" could be added in the CRD definition to group the start and stop of all test cases.

In addition, all the necessary changes should be added to the existing code to take into account this new field and to change the operator logic depending on whether this field is initialised.

[yorugac]

Hi @ivanape! Thanks for opening the issue 🙂 A clarification on usage:

The operator should treat this set of tests as a unit and perform all start and stop operations together.

Do you mean here that the tests should start at the same time or sequentially, one after another? I imagine one could need both, depending on the use case.

Also, there's quite a bit of related discussion in k6 issue: grafana/k6#1342. From your viewpoint, what is the difference from that issue?

[yorugac]

Related forum question:
https://community.grafana.com/t/v0-0-14-testrun-property-spec-paused-not-behaving-as-expected/124255

Orchestrate the serial execution of a set of TestRun resources. If one of the TestRun executions within the set fails, optionally terminate execution of the set of tests.

Within GitOps context.