Add script to make graphs from problem size sweeps #245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It can now take strings with spaces as executables. This allows you to treat things like "srun -n8 ./bin/raja_perf.exe" as an executable.
This helps with graphing as they are used in naming graphs
This helps differentiate different tunings and sweeps
|
YES! If its ready do you want to mark it ready for review? |
rhornung67
requested changes
May 13, 2022
|
There are still some rough edges in this script and I want to add some more comments. |
This allows you to make new kinds of data from other kinds without having to write every kind out beforehand, like avg<time(s)> and avg<FLOPS>.
Now plotting multiple kinds of data will appear on the same graph Added support for linear regression and segmented linear regression
Now you can reduce over arbitrary axes for example min<time(s)>[tuning_index] will yield the min time across all tunings.
Only finished bar graphs so far
Add tiny amount
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Simplify creating graphs from problem size sweeps
Add a script that can create graphs from the output of our problem size sweep script.
Example workflow
Collecting data with sweep_size.sh
The sweep_size.sh script lets you run the perf suite and collect data into a file layout that the plotting script expects.
Here is an example invocation that I made on rzvernal, an ElCap early access machine with 8 gpus per node. This sweep ran with problem sizes 1024, 2048, …, 83886008 and output to a rzvernal_4.5.2 directory because I was testing the rocm 4.5.2 compiler. The arguments after – are passed directly to raja perf, I excluded the ATOMIC and INDEXLIST kernels due to long runtimes.
Plotting data with sweep_graph.py
The sweep_graph.py script lets you graph the data collected by sweep_size.sh.
Here is an example of making line graphs of bandwidth vs problem size for every kernel.
Here is an example of making a bar graph of bandwidth for all kernels and variants at the 8M problem size. Here the max operator us used to pick the max bandwidth of all tunings, collapsing out the tuning dimension.
Here is an example of making a bar graph of speedup for all kernels and variants at the 8M problem size vs the RAJA_CUDA variant on rzansel. The min here is used to pick the min runtime of all tunings, collapsing out the tuning dimension.