Does cargo bench work with flamegraph?

[blaesus]

I have benchmarks set up in benches directory, connected to the main project with a [[bench]] section in Cargo.toml. The benches are simple function calls invoked by their own main(), without Criterion or other benchmark-related dependencies.

Currently we simply run cargo bench to compile benchmarks, which produces a benchmark_name_hash executable file in target/release, for which we manually draw flamegraph by with

flamegraph benchmark_name_hash

This produces usable flamegrpahs.

However, can this be simplified?

We tried this command:

cargo flamegraph --bin=benchmark-name

It doesn't work. Error message is

error: no bin target named `benchmark-name`


WARNING: building without debuginfo. Enable symbol information by adding the following lines to Cargo.toml:

[profile.release]
debug = true

Adding the profile.release section does not help.

Operating system is OSX 10.14.

[jonas-schievink]

Hi @blaesus, cargo flamegraph now supports a --bench argument. Does this do what you expect?

[jaseemabid]

@jonas-schievink There are 6 other issues related to benchmark support and some of them have been closed already - but I've been unable to get benches work so far.

To clarify, does cargo-flamegraph support libraries with no binaries with the current master?

I've verified that $ cargo bench <name> works as expected.

Here are all the different combinations I've tried and failed:

$ cargo flamegraph --dev --bench name
error: no bench target named `name`
cargo build failed: None

$ cargo flamegraph --dev --bench .
error: no bench target named `.`
cargo build failed: None

$ cargo flamegraph bench <name>
   Compiling codes v0.1.0 (/home/j/src/codes)
    Finished release [optimized + debuginfo] target(s) in 1.61s
no Rust binary targets found

$ lamegraph --dev --bench <name>
error: Found argument '--dev' which wasn't expected, or isn't valid in this context

$ flamegraph  --bench <name>
error: Found argument '--bench' which wasn't expected, or isn't valid in this context

$ flamegraph bench <name>
Workload failed: No such file or directory
failed to sample program

$ flamegraph bench .
Workload failed: No such file or directory
failed to sample program

Can you provide an example command please?

[jonas-schievink]

@jaseemabid on what code are you running that? cargo flamegraph --bench name is the right syntax, but you need to provide a benchmark name that actually exists.

I got this part working locally, but I'm getting "Workload failed: Permission denied" when it tries to profile it.

[jaseemabid]

@jonas-schievink I'm passing the same name to both cargo-flamegraph and flamegraph that I passed to cagro bench, it's probably using some other kind of name matching semantics? I've tried all ways you could possibly reference a module, but I keep getting the same error error: no bench target named foo::bar back.

Is there a command to run all benches instead?

I'll probably take a look at the code later because I cant make any progress with the current error messages from the cli, but it would have been really nice to make this work once before jumping in.

[jaseemabid]

@jonas-schievink I had a whole bunch of permission errors on mac, but that's mostly osx issues instead of cargo-flamegraph. Switched to linux instead and now I'm facing this.

[jonas-schievink]

cargo bench doesn't run on modules, it uses source files or directories placed in the benches directory. I'm afraid I can't really help you without having access to the source code you're trying to profile.

[jaseemabid]

Moving the benchmarks explicitly to benches/ folder did the trick, I had them with the rest of the unit tests. Thanks @jonas-schievink

[jonas-schievink]

Closing as fixed.