Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike bookmark: Show deviation from reported averaged #23

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

garethbragg
Copy link
Contributor

@garethbragg garethbragg commented Feb 16, 2024

Some folk are interested in more context around the averaged values these charts show - specifically, how far can we expect these metrics to stray from their reported averages? Just what are those averages hiding?

Spent a little time hacking away and got this far:

image

Recording this work on a draft PR to keep my notes close to the work, but anyone's welcome to comment if they're interested.

Some shenanigans to remember if I come back to this.

Maths
Deviation is not standardised here! We're pulling through actual high/low values that feed in to the averaged data points, and visualising those.

Google charts choices
This is using the intervals feature of Google Charts line charts - in fact, it's a suggested use case for that feature.

There are other visualisation options, which may help with a problem below, but I like this one.

MTTR
Applying the same pattern to MTTR feels doable, but I hit an error rendering the chart - I think this is due to having no MTTR data in the chart I was testing against, but haven't confirmed this hypothesis.

The error isn't exactly helpful, but my guess is we're asking to draw an area with no vertices, and that's causing issues.

If that's true, any chart (with this deviation interval thing) will fail to render if there are no data points at all.

This is different behaviour from current charts having no data at all, as shown by the current MTTR charts.

CFR
Not sure how this pattern can be (meaningfully) applied to CFR, since it's already a proportional measure - something like a candlestick visualisation could be relevant, but is it useful?

Y Axis
I've occasionally had the minimum value on the Y-Axis go below 0 with these changes in place - not looked into why, but I should!

Data bloat, graph crowding, and future direction
This relies on cramming more data into our existing BucketedMetrics - I think that's OK, but does feel a bit icky.

It also makes the charts somewhat noisier. How easy would it be to make showing deviation data optional? Relevant SO link: https://stackoverflow.com/questions/72010311/show-hide-toggle-data-series-on-google-chart-by-clicking-legend

I've also looked into adding chart areas to show performance against whatever the latest State of DevOps report considers 'high' performance (etc.). Having that info and this on the same chart could become difficult to read. Worth considering options here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant