Skip to content

Commit

Permalink
tracing: reorganize benchmarks for comparability (tokio-rs#2178)
Browse files Browse the repository at this point in the history
Currently, `tracing`'s benchmark suite benchmarks the same behaviors
(e.g. creating a span, recording an event, etc) across a handful of
cases: with no default dispatcher, with a global default, and with a
scoped (thread-local) default. We use criterion's `benchmark_group` to
represent each kind of dispatcher, and `bench_function` for each
behavior being measured.

This is actually kind of backwards relative to how Criterion is
_supposed_ to be used. `bench_function` is intended for comparing
different implementations of the *same* thing, with the
`benchmark_group` representing what's being compared. If we inverted the
structure of these benchmarks, Criterion would give us nicer plots that
would allow comparing the performance of each dispatch type on the same
task.

This PR reorganizes the benchmarks so that each behavior being tested
(such as entering a span or recording an event) is a `benchmark_group`,
and each dispatch type (none, global, or scoped) is a `bench_function`
within that group. Now, Criterion will generate plots for each group
comparing the performance of each dispatch type in that benchmark.

For example, we now get nice comparisons like this:
![image](https://user-images.githubusercontent.com/2796466/175659314-835664ac-a8cf-4b07-91ee-f8cfee77bfbb.png)

Unfortunately, this required splitting each benchmark type out into its
own file. This is because, once we set the global default dispatcher
within one benchmark group, it would remain set for the entire lifetime
of the process --- there would be no way to test something else with no
global default. But, I think this is fine, even though it means we now
have a bunch of tiny files: it also allows us to run an individual
benchmark against every combination of dispatch types, without having to
run unrelated benches. This is potentially useful if (for example)
someone is changing only the code for recording events, and not spans.
  • Loading branch information
hawkw authored and kaffarell committed May 22, 2024
1 parent 79e203f commit adf9eb3
Show file tree
Hide file tree
Showing 4 changed files with 25 additions and 427 deletions.
26 changes: 25 additions & 1 deletion tracing/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,31 @@ name = "span_repeated"
harness = false

[[bench]]
name = "global_subscriber"
name = "dispatch_get_ref"
harness = false

[[bench]]
name = "empty_span"
harness = false

[[bench]]
name = "enter_span"
harness = false

[[bench]]
name = "event"
harness = false

[[bench]]
name = "span_fields"
harness = false

[[bench]]
name = "span_no_fields"
harness = false

[[bench]]
name = "span_repeated"
harness = false

[badges]
Expand Down
136 changes: 0 additions & 136 deletions tracing/benches/global_subscriber.rs

This file was deleted.

101 changes: 0 additions & 101 deletions tracing/benches/no_subscriber.rs

This file was deleted.

Loading

0 comments on commit adf9eb3

Please sign in to comment.