Add granular data to the HTML report graphs #415

slashrsm · 2022-01-05T17:17:20Z

Closes #410.

I used the default color scheme for now. I don't particularly like it, but I didn't have any better idea either.

jeremyandrews

I'm very excited to see this coming together! It'll provide much more insight into load test results.

src/graph.rs

jeremyandrews · 2022-02-11T08:17:33Z

src/graph.rs

-    pub(crate) fn get_tasks_per_second_graph(&self) -> Graph<usize> {
-        self.create_graph_from_data("graph-tps", "Tasks #", &self.tasks_per_second)
+    pub(crate) fn get_tasks_per_second_graph(&self) -> Graph<usize, usize> {
+        self.create_graph_from_single_data("graph-tps", "Tasks #", self.tasks_per_second.clone())


Why is tasks_per_second using single_data? It's useful to see more granular data if possible.

Based on what should we divide tasks?

jeremyandrews · 2022-02-11T08:30:46Z

src/graph.rs

+            let mut legend = vec!["Total"];
+
+            let mut other_values = String::new();
+            for (title, sub_data) in self.data.iter() {


Can we sort these by #, so for example the request that makes the most requests in the graph comes first in this legend, the request that makes the second most requests come second, and so on? Otherwise I'm finding it very difficult to match of lines with the legend and to enable/disable specific elements.

(As a reference: this is sorting by name, but likely we can use the same general mechanism to sort by #: https://github.com/tag1consulting/goose/blob/main/src/metrics.rs#L985)

jeremyandrews · 2022-02-11T08:32:55Z

src/graph.rs

@@ -331,7 +382,7 @@ impl<'a, T: Serialize> Graph<'a, T> {
                    var myChart = echarts.init(chartDom);

                    myChart.setOption({{
-                        color: ['#2c664f'],
+                        color: ['#2c664f', '#5470c6', '#91cc75', '#fac858', '#ee6666', '#73c0de', '#3ba272', '#fc8452', '#9a60b4', '#ea7ccc'],


So if there are more than 10 elements on the graph, do we re-use the colors?

Correct. We could add more, but at some point they will start repeating anyway.

Maybe there is a way to set these programmatically? A follow-up maybe?

👍 Followup works. Can you create an issue for this?

src/graph.rs

jeremyandrews · 2022-02-11T08:40:53Z

How difficult would it be to add a flag to disable this verbose data? It creates much larger report files, and perhaps some would prefer the simpler and smaller graphs that we've generated up to this PR? I'd leave granular data enabled by default, but it could be useful to have an option like --no-granular-data or something, especially useful on larger/longer load tests?

nicompte · 2022-03-16T14:22:52Z

Hello there, just a suggestion after testing your branch, do you think it would be possible to keep the same color for the same request, between two runs?

…data colletion and display.

… metric.

…time_per_second metric.

…_requests_per_second() and graph::record_errors_per_second().

…ing it.

slashrsm · 2022-03-24T17:11:08Z

I added --no-granular-data configuration option.

Hello there, just a suggestion after testing your branch, do you think it would be possible to keep the same color for the same request, between two runs?

Good point. I added ordering to the lines/legend based on the numbers behind them. I suspect that same order also means same colors so this might solve your suggestion too. @nicompte Do you mind testing a bit?

nicompte · 2022-03-25T14:37:48Z

Good point. I added ordering to the lines/legend based on the numbers behind them. I suspect that same order also means same colors so this might solve your suggestion too. @nicompte Do you mind testing a bit?

Works great thanks!

jeremyandrews · 2022-04-05T11:03:42Z

src/config.rs

@@ -59,6 +59,7 @@ const DEFAULT_PORT: &str = "5115";
 /// --no-error-summary         Doesn't display an error summary
 /// --no-print-metrics         Doesn't display metrics at end of load test
 /// --report-file NAME         Create an html-formatted report
+/// --no-granular-data         Disables granular data in HTML report graphs


Would --no-granular-report make it more clear that this specifically impacts --report-file and no other data?

jeremyandrews · 2022-04-05T11:08:52Z

src/graph.rs

-    /// represents one second.
-    users_per_second: Vec<usize>,
+    /// Counts requests per second for each request type.
+    requests_per_second: HashMap<String, TimeSeries<u32, u32>>,


(Followup is fine) it seems worthwhile to create a custom Type for HashMap<String, TimeSeries<u32, u32>>, something like ItemsPerSecond or PerSecondCounter or ...? I see it throughout the new code, and we'd document it thoroughly where the type is created.

jeremyandrews

This is great, thanks! I'm doing more testing, but there's nothing blocking a merge now. I think the configuration option change is worthwhile to be more clear

slashrsm force-pushed the 410_granular_graph_metrics branch from d0ec6c8 to 1f1f6ac Compare January 21, 2022 09:26

slashrsm force-pushed the 410_granular_graph_metrics branch from bd61dc6 to c2908ac Compare February 10, 2022 12:08

slashrsm marked this pull request as ready for review February 10, 2022 20:36

slashrsm requested a review from jeremyandrews February 10, 2022 20:36

jeremyandrews requested changes Feb 11, 2022

View reviewed changes

slashrsm force-pushed the 410_granular_graph_metrics branch from 32f17fb to 37babb5 Compare March 17, 2022 09:57

slashrsm added 6 commits March 23, 2022 16:58

tag1consulting#410: Update graph APIs to be able to support granular …

69d472f

…data colletion and display.

tag1consulting#410: Add more documentation.

216410b

tag1consulting#410: Collect granular graph data for errors_per_second…

20c02c8

… metric.

tag1consulting#410: Collect granular graph data for average_response_…

3222223

…time_per_second metric.

tag1consulting#410: Display aggregated metrics that we collected.

35206a2

tag1consulting#410: Optimize string arguments sent into graph::record…

feed181

…_requests_per_second() and graph::record_errors_per_second().

slashrsm force-pushed the 410_granular_graph_metrics branch from 37babb5 to 7478a19 Compare March 23, 2022 15:59

tag1consulting#410: Graph elements ordering - first set of basic APIs.

d1a0b6d

slashrsm force-pushed the 410_granular_graph_metrics branch from 7478a19 to d1a0b6d Compare March 23, 2022 16:02

slashrsm added 3 commits March 23, 2022 21:17

tag1consulting#410: Sort graph lines.

f04ca50

tag1consulting#410: Maintain time series total instead of re-calculat…

b6c2edf

…ing it.

tag1consulting#410: Rename functions to be more consistent.

bebb493

slashrsm requested a review from jeremyandrews March 24, 2022 17:11

tag1consulting#410: Add --no-granular-data configuration option.

0e8ce09

slashrsm force-pushed the 410_granular_graph_metrics branch from 7fb42a6 to 0e8ce09 Compare March 24, 2022 17:57

slashrsm added 2 commits March 28, 2022 09:42

tag1consulting#410: Test graphs with granular data disabled.

8afb832

tag1consulting#410: Add changelog entry.

174d8d9

jeremyandrews reviewed Apr 5, 2022

View reviewed changes

jeremyandrews approved these changes Apr 5, 2022

View reviewed changes

jeremyandrews merged commit 6ad9b12 into tag1consulting:main Apr 5, 2022

This was referenced Apr 8, 2022

Programatically generate color scheme for HTML report graphs. #428

Open

Create new "ItemsPerSecond" data type for graph data collection #429

Open

Rename --no-granular-data to --no-granular-report #430

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add granular data to the HTML report graphs #415

Add granular data to the HTML report graphs #415

slashrsm commented Jan 5, 2022 •

edited

Loading

jeremyandrews left a comment

jeremyandrews Feb 11, 2022

slashrsm Mar 24, 2022

jeremyandrews Apr 5, 2022

jeremyandrews Feb 11, 2022

jeremyandrews Feb 11, 2022

slashrsm Mar 23, 2022

jeremyandrews Feb 11, 2022

slashrsm Mar 14, 2022

jeremyandrews Mar 14, 2022

jeremyandrews commented Feb 11, 2022

nicompte commented Mar 16, 2022

slashrsm commented Mar 24, 2022

nicompte commented Mar 25, 2022

jeremyandrews Apr 5, 2022

jeremyandrews Apr 5, 2022

jeremyandrews left a comment

Add granular data to the HTML report graphs #415

Add granular data to the HTML report graphs #415

Conversation

slashrsm commented Jan 5, 2022 • edited Loading

jeremyandrews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremyandrews commented Feb 11, 2022

nicompte commented Mar 16, 2022

slashrsm commented Mar 24, 2022

nicompte commented Mar 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremyandrews left a comment

Choose a reason for hiding this comment

slashrsm commented Jan 5, 2022 •

edited

Loading