Document bootstrap integration with rustc-perf (rust-lang#2005)

src/SUMMARY.md

@@ -36,6 +36,7 @@
 - [Profiling the compiler](./profiling.md)
     - [with the linux perf tool](./profiling/with_perf.md)
     - [with Windows Performance Analyzer](./profiling/wpa_profiling.md)
+    - [with the Rust benchmark suite](./profiling/with_rustc_perf.md)
 - [crates.io Dependencies](./crates-io.md)
 
 # Contributing to Rust

src/profiling/with_perf.md

@@ -54,46 +54,23 @@ you made in the beginning. But there are some things to be aware of:
 
 ### Gathering a perf profile from a `perf.rust-lang.org` test
 
-Often we want to analyze a specific test from `perf.rust-lang.org`. To
-do that, the first step is to clone
-[the rustc-perf repository][rustc-perf-gh]:
+Often we want to analyze a specific test from `perf.rust-lang.org`.
+The easiest way to do that is to use the [rustc-perf][rustc-perf]
+benchmarking suite, this approach is described [here](with_rustc_perf.md).
 
-```bash
-git clone https://github.com/rust-lang/rustc-perf
-```
-
-[rustc-perf-gh]: https://github.com/rust-lang/rustc-perf
-
-#### Doing it the easy way
-
-Once you've cloned the repo, you can use the `collector` executable to
-do profiling for you! You can find
-[instructions in the rustc-perf readme][rustc-perf-readme].
-
-[rustc-perf-readme]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md#profiling
-
-For example, to measure the clap-rs test, you might do:
+Instead of using the benchmark suite CLI, you can also profile the benchmarks manually. First,
+you need to clone the [rustc-perf][rustc-perf] repository:
 
 ```bash
-./target/release/collector                                      \
-  --output-repo /path/to/place/output                           \
-  profile perf-record                                           \
-  --rustc /path/to/rustc/executable/from/your/build/directory   \
-  --cargo `which cargo`                                         \
-  --filter clap-rs                                              \
-  --builds Check                                                \
+$ git clone https://github.com/rust-lang/rustc-perf
 ```
 
-You can also use that same command to use cachegrind or other profiling tools.
-
-#### Doing it the hard way
-
-If you prefer to run things manually, that is also possible. You first
-need to find the source for the test you want. Sources for the tests
+and then find the source code of the test that you want to profile. Sources for the tests
 are found in [the `collector/compile-benchmarks` directory][compile-time dir]
 and [the `collector/runtime-benchmarks` directory][runtime dir]. So let's
 go into the directory of a specific test; we'll use `clap-rs` as an example:
 
+[rustc-perf]: https://github.com/rust-lang/rustc-perf
 [compile-time dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/compile-benchmarks
 [runtime dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/runtime-benchmarks
 

src/profiling/with_rustc_perf.md

@@ -0,0 +1,37 @@
+# Profiling with rustc-perf
+
+The [Rust benchmark suite][rustc-perf] provides a comprehensive way of profiling and benchmarking
+the Rust compiler. You can find instructions on how to use the suite in its [manual][rustc-perf-readme].
+
+However, using the suite manually can be a bit cumbersome. To make this easier for `rustc` contributors,
+the compiler build system (`bootstrap`) also provides built-in integration with the benchmarking suite,
+which will download and build the suite for you, build a local compiler toolchain and let you profile it using a simplified command-line interface.
+
+You can use the `./x perf -- <command> [options]` command to use this integration.
+
+> Note that you need to specify arguments after `--` in the `x perf` command! You will not be able to pass arguments without the double dashes.
+
+You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `config.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler.
+
+`x perf` currently supports the following commands:
+- `benchmark <id>`: Benchmark the compiler and store the results under the passed `id`.
+- `compare <baseline> <modified>`: Compare the benchmark results of two compilers with the two passed `id`s.
+- `eprintln`: Just run the compiler and capture its `stderr` output. Note that the compiler normally does not print
+  anything to `stderr`, you might want to add some `eprintln!` calls to get any output. 
+- `samply`: Profile the compiler using the [samply][samply] sampling profiler.
+- `cachegrind`: Use [Cachegrind][cachegrind] to generate a detailed simulated trace of the compiler's execution.
+
+> You can find a more detailed description of the profilers in the [`rustc-perf` manual][rustc-perf-readme-profilers].
+
+You can use the following options for the `x perf` command, which mirror the corresponding options of the
+`profile_local` and `bench_local` commands that you can use in the suite:
+
+- `--include`: Select benchmarks which should be profiled/benchmarked.
+- `--profiles`: Select profiles (`Check`, `Debug`, `Opt`, `Doc`) which should be profiled/benchmarked.
+- `--scenarios`: Select scenarios (`Full`, `IncrFull`, `IncrPatched`, `IncrUnchanged`) which should be profiled/benchmarked.
+
+[samply]: https://github.com/mstange/samply
+[cachegrind]: https://www.cs.cmu.edu/afs/cs.cmu.edu/project/cmt-40/Nice/RuleRefinement/bin/valgrind-3.2.0/docs/html/cg-manual.html
+[rustc-perf]: https://github.com/rust-lang/rustc-perf
+[rustc-perf-readme]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md
+[rustc-perf-readme-profilers]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md#profiling-local-builds

src/tests/perf.md

@@ -17,6 +17,9 @@ configurations include "fresh builds", builds with incremental compilation, etc.
 The result of a perf run is a comparison between two versions of the compiler
 (by their commit hashes).
 
+You can also use `rustc-perf` to manually benchmark and profile the compiler
+[locally](../profiling/with_rustc_perf.md).
+
 ### Automatic perf runs
 
 After every PR is merged, a suite of benchmarks are run against the compiler.