Support running jmh profilers #217
Conversation
| | Option | Description | Possible Values | Default Value | | ||
| |---------------------------------------------|--------------------------------------------------------|----------------------------------------|----------------------| | ||
| | `advanced("jvmForks", value)` | Specifies the number of times the harness should fork. | Non-negative Integer, `"definedByJmh"` | `1` | | ||
| | `advanced("jvmProfiler", "value")` | Specifies the profiler to be used during benchmarking. | String identifier of the profiler | `null` (No profiler) | |
There was a problem hiding this comment.
How one can specify profiler's options?
Something like advance("jvmProfiler", "gc:churn=true") fails with:
Exception in thread "main" java.lang.IllegalStateException: Parameter name and value format is required for advanced.
at kotlinx.benchmark.RunnerConfiguration.mapSingleValues(RunnerConfiguration.kt:58)
at kotlinx.benchmark.RunnerConfiguration.<init>(RunnerConfiguration.kt:28)
at kotlinx.benchmark.jvm.JvmBenchmarkRunnerKt.main(JvmBenchmarkRunner.kt:17)
|
|
||
| **Notes on "jvmForks":** | ||
| - **0** - "no fork", i.e., no subprocesses are forked to run benchmarks. | ||
| - A positive integer value – the amount used for all benchmarks in this configuration. | ||
| - **"definedByJmh"** – Let JMH determine the amount, using the value in the [`@Fork` annotation](https://javadoc.io/doc/org.openjdk.jmh/jmh-core/latest/org/openjdk/jmh/annotations/Fork.html) for the benchmark function or its enclosing class. If not specified by `@Fork`, it defaults to [Defaults.MEASUREMENT_FORKS (`5`)](https://javadoc.io/doc/org.openjdk.jmh/jmh-core/latest/org/openjdk/jmh/runner/Defaults.html#MEASUREMENT_FORKS). | ||
|
|
||
| **Notes on "jvmProfiler":** | ||
| - This option corresponds to the `-prof` command line option when running JMH benchmarks from console. | ||
| Some examples of possible values include `gc`, `stack`, `cl`, `perf`, and `dtraceasm`. |
There was a problem hiding this comment.
Currently, for asm-annotating profilers (perfasm, dtraceasm, xctraceasm), users won't be able to obtain meaningful results if hsdis is not on the dynamic loader's library path.
On macOS, you can't put a library into /usr/lib and dyld won't load libraries from other locations (like /usr/local/lib) unless they are explicitly specified in DYLD_LIBRARY_PATH.
And it seems like we don't setup DYLD_LIBRARY_PATH for the runner's process (by copying its value from the parent process).
We may consider bypassing library path to the runner process or suggest users to build benchmark's JAR and use it directly, when perfasm profilers are needed.
Additionally, improve output format and include secondary results.
Resolves #50