Displaying thresholds

[oleiade]

I cannot remember on top of my head if you got the threshold definition from the k6 output, so please feel free to dismiss this if you don't receive it.

If we do, though, I would find it very helpful to display the thresholds in a table and their current status (as we did in the k6 saas website). I could even imagine having a "threshold line" on the related chart (http_req_duration: p(95) < 200ms leading to a tiny colored horizontal line on the relevant HTTP chart).

Once again this is highly hypothetical, feel free to dismiss it if not relevant 👍🏻

[szkiba]

I can get thresholds (I do it one of my extensions) and yes it seem good idea to display it.
Thank you.

[oleiade]

This requires quite some work to evaluate thresholds and display a dedicated line.
Need a decision on whether we want to do that short-term or not.

[ErikHaaijen]

It would be cool to see a (red) line to show the thresholds together with the results.

[Caritrian]

Or a green line ;) But a general avaliablity of a threshold line would be cool

[PabloDelBarrioArnanz]

I like the idea and maybe it would also be nice to have a specific graph?

[szkiba]

The event stream already contains the result of the threshold evaluation (threshold event), a threshold event is sent at the end of each update period (if there was a failed threshold). This event contains the name of the metric and the threshold expression that failed.

I recommend displaying it on the Summary tab, with a symbol before the name of the metric (❌ if failed, ✅ if threshold expression is configured and successful, empty if there is no threshold expression, these symbols are just examples, FYI @2Steaks).

I think the display on the graph is not trivial:

First, the k6 thresholds use aggregates calculated for the entire test run time. The graphs of the web dashboard display the aggregates calculated for the update period time (10s by default). That is, the aggregate value of the metric for the update period time and the threshold value of the metric for the entire test run time should be displayed on a graph. In other words, a snapshot-type value and a cumulative-type value would appear on the same graph. I think that would be confusing.

On the other hand, many graphs contain multiple series. They are either completely different metrics or different aggregates of the same metric. It is not easy to visually display one or more threshold lines on these graphs without being confusing.

I think displaying thresholds is a good idea, but visual design can be a challenge.