Spike bookmark: Show deviation from reported averaged #23
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some folk are interested in more context around the averaged values these charts show - specifically, how far can we expect these metrics to stray from their reported averages? Just what are those averages hiding?
Spent a little time hacking away and got this far:
Recording this work on a draft PR to keep my notes close to the work, but anyone's welcome to comment if they're interested.
Some shenanigans to remember if I come back to this.
Maths
Deviation is not standardised here! We're pulling through actual high/low values that feed in to the averaged data points, and visualising those.
Google charts choices
This is using the intervals feature of Google Charts line charts - in fact, it's a suggested use case for that feature.
There are other visualisation options, which may help with a problem below, but I like this one.
MTTR
Applying the same pattern to MTTR feels doable, but I hit an error rendering the chart - I think this is due to having no MTTR data in the chart I was testing against, but haven't confirmed this hypothesis.
The error isn't exactly helpful, but my guess is we're asking to draw an area with no vertices, and that's causing issues.
If that's true, any chart (with this deviation interval thing) will fail to render if there are no data points at all.
This is different behaviour from current charts having no data at all, as shown by the current MTTR charts.
CFR
Not sure how this pattern can be (meaningfully) applied to CFR, since it's already a proportional measure - something like a candlestick visualisation could be relevant, but is it useful?
Y Axis
I've occasionally had the minimum value on the Y-Axis go below 0 with these changes in place - not looked into why, but I should!
Data bloat, graph crowding, and future direction
This relies on cramming more data into our existing
BucketedMetrics
- I think that's OK, but does feel a bit icky.It also makes the charts somewhat noisier. How easy would it be to make showing deviation data optional? Relevant SO link: https://stackoverflow.com/questions/72010311/show-hide-toggle-data-series-on-google-chart-by-clicking-legend
I've also looked into adding chart areas to show performance against whatever the latest State of DevOps report considers 'high' performance (etc.). Having that info and this on the same chart could become difficult to read. Worth considering options here.