Skip to content

Commit

Permalink
release: 0.3.1
Browse files Browse the repository at this point in the history
  • Loading branch information
joshstoik1 committed Aug 23, 2022
2 parents 1047dea + 0b5ea52 commit 515b3a6
Show file tree
Hide file tree
Showing 10 changed files with 285 additions and 125 deletions.
18 changes: 17 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,25 @@
# Changelog


## [0.3.1](https://github.com/Blobfolio/brunch/releases/tag/v0.3.1) - 2022-08-23

### Changes

* Coloration tweaks;
* Improved float handling;
* Suppress _Change_ summary column when there aren't any;
* Warn if the same `Bench` name is submitted twice;

### Fixed

* env `NO_BRUNCH_HISTORY` should only apply when `=1`;
* Normalize whitespace in `Bench` names to prevent display weirdness;



## [0.3.0](https://github.com/Blobfolio/brunch/releases/tag/v0.3.0) - 2022-08-22

This release includes a number of improvements to the `Brunch` API, but as a result, existing benchmarks will need a few (minor) changes when migrating from `0.2.x` to `0.3.x`.
This release includes a number of improvements to the `Brunch` API, but also some **breaking changes**. Existing benchmarks will require a few (minor) adjustments when migrating from `0.2.x` to `0.3.x`.

First and foremost, `Bench::new` has been streamlined, and now takes the name as a single argument (rather than two). When migrating, just glue the two values back together, e.g. `"foo::bar", "baz(20)"` to `"foo::bar::baz(20)"`.

Expand Down
4 changes: 2 additions & 2 deletions CREDITS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Project Dependencies
Package: brunch
Version: 0.3.0
Generated: 2022-08-22 19:57:13 UTC
Version: 0.3.1
Generated: 2022-08-23 19:16:04 UTC

| Package | Version | Author(s) | License |
| ---- | ---- | ---- | ---- |
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "brunch"
version = "0.3.0"
version = "0.3.1"
authors = ["Blobfolio, LLC. <hello@blobfolio.com>"]
edition = "2021"
rust-version = "1.61"
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@
[![crates.io](https://img.shields.io/crates/v/brunch.svg)](https://crates.io/crates/brunch)
[![Build Status](https://github.com/Blobfolio/brunch/workflows/Build/badge.svg)](https://github.com/Blobfolio/brunch/actions)
[![Dependency Status](https://deps.rs/repo/github/blobfolio/brunch/status.svg)](https://deps.rs/repo/github/blobfolio/brunch)

[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/Blobfolio/brunch)



`Brunch` is a very simple Rust micro-benchmark runner inspired by [`easybench`](https://crates.io/crates/easybench). It has roughly a million times fewer dependencies than [`criterion`](https://crates.io/crates/criterion), does not require nightly, and maintains a "last run" state so can show relative changes benchmark-to-benchmark.

The formatting is also quite pretty.
(The formatting is also quite pretty.)

As with all Rust benchmarking, there are a lot of caveats, and results might be artificially fast or slow. For best results:
* build optimized;
* collect lots of samples;
* repeat identical runs to get a feel for the natural variation;
* Build optimized;
* Collect lots of samples;
* Repeat identical runs to get a feel for the natural variation;

`Brunch` cannot measure time below the level of a nanosecond, so if you're trying to benchmark methods that are _really_ fast, you may need to wrap them in a method that runs through several iterations at once. For example:

Expand Down Expand Up @@ -77,7 +77,7 @@ harness = false

The following optional environmental variables are supported:

* `NO_BRUNCH_HISTORY`: don't save or load run-to-run history data;
* `NO_BRUNCH_HISTORY=1`: don't save or load run-to-run history data;
* `BRUNCH_DIR=/some/directory`: save run-to-run history data to this folder instead of `std::env::temp_dir`;


Expand Down
146 changes: 111 additions & 35 deletions src/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,13 @@ impl Benches {
/// // Repeat push as needed.
/// benches.finish();
/// ```
pub fn push(&mut self, b: Bench) { self.0.push(b); }
pub fn push(&mut self, mut b: Bench) {
if ! b.is_spacer() && self.has_name(&b.name) {
b.stats.replace(Err(BrunchError::DupeName));
}

self.0.push(b);
}

/// # Finish.
///
Expand Down Expand Up @@ -105,8 +111,14 @@ impl Benches {
// Build the summaries.
let mut history = History::default();
let mut summary = Table::default();
let names: Vec<Vec<char>> = self.0.iter()
.filter_map(|b|
if b.is_spacer() { None }
else { Some(b.name.chars().collect()) }
)
.collect();
for b in &self.0 {
summary.push(b, &history);
summary.push(b, &names, &history);
}

// Update the history.
Expand All @@ -129,6 +141,13 @@ impl Benches {
}
}

impl Benches {
/// # Has Name.
fn has_name(&self, name: &str) -> bool {
self.0.iter().any(|b| b.name == name)
}
}



#[derive(Debug)]
Expand Down Expand Up @@ -175,8 +194,27 @@ impl Bench {
let name = name.as_ref().trim();
assert!(! name.is_empty(), "Name is required.");

// Compact and normalize whitespace, but otherwise pass whatever the
// name is on through.
let mut ws = false;
let name = name.chars()
.filter_map(|c|
if c.is_whitespace() {
if ws { None }
else {
ws = true;
Some(' ')
}
}
else {
ws = false;
Some(c)
}
)
.collect();

Self {
name: name.to_owned(),
name,
samples: DEFAULT_SAMPLES,
timeout: DEFAULT_TIMEOUT,
stats: None,
Expand Down Expand Up @@ -426,10 +464,10 @@ impl Default for Table {
fn default() -> Self {
Self(vec![
TableRow::Normal(
"\x1b[1;38;5;13mMethod".to_owned(),
"\x1b[1;95mMethod".to_owned(),
"Mean".to_owned(),
"Change".to_owned(),
"Samples\x1b[0m".to_owned()
"Samples".to_owned(),
"Change\x1b[0m".to_owned(),
),
TableRow::Spacer,
])
Expand All @@ -440,13 +478,17 @@ impl fmt::Display for Table {
#[allow(clippy::many_single_char_names)]
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
// Maximum column widths.
let (w1, w2, w3, w4) = self.lens();
let (w1, w2, w3, mut w4) = self.lens();
let changes = self.show_changes();
let width =
if changes { w1 + w2 + w3 + w4 + 12 }
else {
w4 = 0;
w1 + w2 + w3 + 8
};

// Pre-generate the full-width spacer content.
let spacer = format!(
"\x1b[38;5;5m{}\x1b[0m\n",
"-".repeat(w1 + w2 + w3 + w4 + 12)
);
let spacer = format!("\x1b[35m{}\x1b[0m\n", "-".repeat(width));

// Pre-generate padding too. We'll slice this to size each time padding
// is needed.
Expand All @@ -456,13 +498,19 @@ impl fmt::Display for Table {
for v in &self.0 {
let (c1, c2, c3, c4) = v.lens();
match v {
TableRow::Normal(a, b, c, d) => writeln!(
TableRow::Normal(a, b, c, d) if changes => writeln!(
f, "{}{} {}{} {}{} {}{}",
a, &pad[..w1 - c1],
&pad[..w2 - c2], b,
&pad[..w3 - c3], c,
&pad[..w4 - c4], d,
)?,
TableRow::Normal(a, b, c, _) => writeln!(
f, "{}{} {}{} {}{}",
a, &pad[..w1 - c1],
&pad[..w2 - c2], b,
&pad[..w3 - c3], c,
)?,
TableRow::Error(a, b) => writeln!(
f, "{}{} \x1b[1;38;5;208m{}\x1b[0m",
a, &pad[..w1 - c1], b,
Expand All @@ -477,23 +525,24 @@ impl fmt::Display for Table {

impl Table {
/// # Add Row.
fn push(&mut self, src: &Bench, history: &History) {
fn push(&mut self, src: &Bench, names: &[Vec<char>], history: &History) {
if src.is_spacer() { self.0.push(TableRow::Spacer); }
else {
let name = format_name(&src.name);
let name = format_name(src.name.chars().collect(), names);
match src.stats.unwrap_or(Err(BrunchError::NoRun)) {
Ok(s) => {
let time = util::format_time(s.mean);
let time = s.nice_mean();
let diff = history.get(&src.name)
.and_then(|h| s.is_deviant(h))
.unwrap_or_else(|| NO_CHANGE.to_owned());
let (valid, total) = s.samples();
let samples = format!(
"\x1b[2m{}\x1b[0;38;5;5m/\x1b[0;2m{}\x1b[0m",
NiceU64::from(s.valid),
NiceU64::from(s.total),
"\x1b[2m{}\x1b[0;35m/\x1b[0;2m{}\x1b[0m",
NiceU64::from(valid),
NiceU64::from(total),
);

self.0.push(TableRow::Normal(name, time, diff, samples));
self.0.push(TableRow::Normal(name, time, samples, diff));
},
Err(e) => {
self.0.push(TableRow::Error(name, e));
Expand All @@ -502,6 +551,16 @@ impl Table {
}
}

/// # Has Changes?
///
/// Returns true if any of the Change columns have a value.
fn show_changes(&self) -> bool {
self.0.iter().skip(2).any(|v|
if let TableRow::Normal(_, _, _, c) = v { c != NO_CHANGE }
else { false }
)
}

/// # Widths.
fn lens(&self) -> (usize, usize, usize, usize) {
self.0.iter()
Expand Down Expand Up @@ -553,22 +612,39 @@ impl TableRow {
#[allow(clippy::option_if_let_else)]
/// # Format Name.
///
/// Style up a benchmark name.
fn format_name(name: &str) -> String {
// Last opening parenthesis?
if let Some(pos) = name.rfind('(') {
// Is there a namespace thing behind it?
if let Some(pos2) = name[..pos].rfind("::") {
format!("\x1b[2m{}::\x1b[0m{}", &name[..pos2], &name[pos2 + 2..])
}
else {
format!("\x1b[2m{}\x1b[0m{}", &name[..pos], &name[pos..])
}
/// Style up a benchmark name by dimming common portions, and highlighting
/// unique ones.
///
/// This approach won't scale well, but the bench count for any given set
/// should be relatively low.
fn format_name(mut name: Vec<char>, names: &[Vec<char>]) -> String {
// Find the first unique char occurrence.
let pos: usize = names.iter()
.filter_map(|other|
if name.eq(other) { None }
else {
name.iter()
.zip(other.iter())
.position(|(l, r)| l != r)
.or_else(|| Some(name.len().min(other.len())))
}
)
.max()
.unwrap_or_default();

if pos == 0 {
"\x1b[94m".chars()
.chain(name.into_iter())
.chain("\x1b[0m".chars())
.collect()
}
// Last namespace thing?
else if let Some(pos) = name.rfind("::") {
format!("\x1b[2m{}::\x1b[0m{}", &name[..pos], &name[pos + 2..])
else {
let b = name.split_off(pos);
"\x1b[34m".chars()
.chain(name.into_iter())
.chain("\x1b[94m".chars())
.chain(b.into_iter())
.chain("\x1b[0m".chars())
.collect()
}
// Leave it boring.
else { ["\x1b[2m", name, "\x1b[0m"].concat() }
}
6 changes: 5 additions & 1 deletion src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ use std::fmt;
///
/// This enum serves as the custom error type for `Brunch`.
pub enum BrunchError {
/// # Duplicate name.
DupeName,

/// # No benches were specified.
NoBench,

Expand All @@ -36,8 +39,9 @@ impl std::error::Error for BrunchError {}
impl fmt::Display for BrunchError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::DupeName => f.write_str("Benchmark names must be unique."),
Self::NoBench => f.write_str("At least one benchmark is required."),
Self::NoRun => f.write_str("Missing \x1b[1;38;5;14mBench::run\x1b[0m."),
Self::NoRun => f.write_str("Missing \x1b[1;96mBench::run\x1b[0m."),
Self::Overflow => f.write_str("Unable to crunch the numbers."),
Self::TooFast => f.write_str("Too fast to benchmark!"),
Self::TooSmall(n) => write!(
Expand Down
12 changes: 6 additions & 6 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@
[![crates.io](https://img.shields.io/crates/v/brunch.svg)](https://crates.io/crates/brunch)
[![Build Status](https://github.com/Blobfolio/brunch/workflows/Build/badge.svg)](https://github.com/Blobfolio/brunch/actions)
[![Dependency Status](https://deps.rs/repo/github/blobfolio/brunch/status.svg)](https://deps.rs/repo/github/blobfolio/brunch)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/Blobfolio/brunch)
`Brunch` is a very simple Rust micro-benchmark runner inspired by [`easybench`](https://crates.io/crates/easybench). It has roughly a million times fewer dependencies than [`criterion`](https://crates.io/crates/criterion), does not require nightly, and maintains a "last run" state so can show relative changes benchmark-to-benchmark.
The formatting is also quite pretty.
(The formatting is also quite pretty.)
As with all Rust benchmarking, there are a lot of caveats, and results might be artificially fast or slow. For best results:
* build optimized;
* collect lots of samples;
* repeat identical runs to get a feel for the natural variation;
* Build optimized;
* Collect lots of samples;
* Repeat identical runs to get a feel for the natural variation;
`Brunch` cannot measure time below the level of a nanosecond, so if you're trying to benchmark methods that are _really_ fast, you may need to wrap them in a method that runs through several iterations at once. For example:
Expand Down Expand Up @@ -71,7 +71,7 @@ harness = false
The following optional environmental variables are supported:
* `NO_BRUNCH_HISTORY`: don't save or load run-to-run history data;
* `NO_BRUNCH_HISTORY=1`: don't save or load run-to-run history data;
* `BRUNCH_DIR=/some/directory`: save run-to-run history data to this folder instead of [`std::env::temp_dir`];
Expand Down
2 changes: 1 addition & 1 deletion src/macros.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ macro_rules! benches {
// Run the benches.
let mut benches = $crate::Benches::default();
$(
::std::eprint!("\x1b[1;38;5;4m\x1b[0m");
::std::eprint!("\x1b[1;34m\x1b[0m");
benches.push($benches);
)+
::std::eprintln!("\n");
Expand Down
Loading

0 comments on commit 515b3a6

Please sign in to comment.