Benchgraph

Easy visualisation of the evolution of your benchmarks.

Wat's benchgraph?

Benchgraph is a tool which helps you to see how your benchmarks evolve through the lifetime of your program.

Why do I need it

Wants to see if a new feature will slow your library down? Wants to know why your users complains that your program starts getting slower? That's what Benchgraph is for.

Enough jabbing, I want nice pictures and buzzwords

All right, all right.

First, here's a picture of a blockchain-generated cat:

Now, here's a screenshot of benchgraph in action on one of our internal benchmark suite:

Wah, I'm convinced! How do I use it?

To use this, just follow these two simple steps:

1. Export your benchmarks results to the format accepted by the benchgraph server
2. Run the server

Export your results

The benchgraph server reads the benchmarks results as an array in nd-json format, where each line contains a record of the form (without the newlines obviously):

{
  "bench_name": "MyBench";
  "commit_rev": "Id of the commit";
  "timestamp": "the date of the commit";
  "time_in_nanos": "Duration of the benchmark";
}

We provide adapters for some benchmarking frameworks (only criterion at the moment, but you're welcome to add more) so that the export is one simple command line.

Run the server

The server is provided as a docker image.

If your benchmark results are all in the benchmarks directory, you can simply run:

docker pull benchgraph/benchgraph:master
docker run \
  -p 8123:8123
  -v $PWD/benchmarks:/benchmarks benchgraph/benchgraph:master \
  /bin/benchgraph /benchmarks