-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pr from fork/137 #254
Pr from fork/137 #254
Changes from all commits
c64db24
2db7cb1
d7ba2ad
940a07e
513455d
e30a1b7
c6e4771
90be795
fc1b920
d681a64
516507d
7914e24
8e08cd5
01d79b7
dc55482
af35eee
130dfb2
b5ad2e4
b1275b7
e89f97d
e7577d1
b19a19f
76aff45
64d5219
0e9a90e
61dae2a
398ba56
c617652
2933af1
1b63fe2
0ce8c63
eb54c42
d8afe7d
29a3873
93faac1
55ed8be
699ff16
2b8b5ce
c256961
85b3d2c
41983b1
074c208
63e1ffd
ced724c
71a9451
dd1bd5c
5dd7d4b
59cc13c
a1e172e
c488212
2098f85
1e3824c
0d3e5be
e2eb0bf
059e4df
4a176b6
a26f65f
0f2709f
8cde548
5beb2b7
9f7a00f
ed0621f
187e7d1
700be35
19ac08a
16b8340
9295555
a40b276
bc0573a
2a761bb
e79cf13
81e29e4
53671b0
a4c12a8
88d98ed
14f0a39
3b2de08
39fde9b
52ac292
a4aec2b
84aedca
248ffd3
8f32df7
763c358
7ec27ce
b73caaa
08ea0f9
052ca63
b8041e7
1950e27
ef58b0c
fffe3e0
4ad948f
ba525df
775701c
b59755e
f5f6a35
f444177
38bc905
ecc8860
0a9bef0
e512b0d
57f50c4
bf45a79
a616fe1
8c118ff
5941881
505cc95
3002f99
10c5480
b19939e
f91cd21
f57e3c7
1cc3f4a
d590132
4800c32
334bd37
2e03612
a353d4f
0171719
7c6f110
9ad36ac
62341ea
1d2e427
93dee34
8985613
dfbdcb9
a742ebe
ef804d4
752f643
ccdfc0e
1ec65b5
d5dbda0
72eb5e9
3ae59ac
22c8f5b
449a705
ed43d87
592c854
dfb6ad3
ca71f90
e8ade72
3b08adc
fc5e6d3
5499c7d
a2708a1
e68710e
a31768e
97edc52
03d2562
49659db
e5bdd6c
401e069
54919cc
b300358
fc2981a
3c6503e
7731dc9
256fc52
70b6cff
e3fc7d7
ed6033a
dbe10a4
aaa3361
cb3742e
42f9619
e5ed98b
5fa11bc
c46858e
09292db
b134ad6
8fd43bd
609549e
6490c18
bd90e75
f010820
6671991
82c0423
5969bf1
e9b1ba0
3993588
b2ec163
aaba7f5
bb52164
6b03dac
e4ca868
06ce8bc
74a4f6e
f2dd795
489c9a5
29dbcdb
b826e28
c3a88ff
cb65f40
7a1f7b9
bd0d585
39e699d
7306808
2eac54f
b7fbc8c
110d578
b29d567
edaa2e2
202635e
548b42a
b831e47
9ed8761
eb96294
203e9ba
5c04627
70371c4
aa34389
dc9ca71
835a7cb
ddc0a04
db6db7b
1954dd8
6d2e1fd
c5bc688
18500fb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -210,3 +210,47 @@ sizes. The CMake option for this is | |||||
|
||||||
will build versions of GPU kernels that use 64, 128, 256, 512, and 1024 threads | ||||||
per GPU thread-block. | ||||||
|
||||||
Building with Caliper | ||||||
--------------------- | ||||||
|
||||||
RAJAPerf Suite may also use Caliper instrumentation, with per variant output into | ||||||
Spot/Hatchet .cali files. Original timing is nested within Caliper annotations and | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
It would be good to add references to Spot and Hatchet also. |
||||||
so is not impacted when Caliper support is turned on. While Caliper is low-overhead | ||||||
it is not zero, so it will add a small amount of timing skew in its data as | ||||||
compared to the original. For much more on Caliper and Hatchet, read their documentation here: | ||||||
| - `Caliper Documentation <http://software.llnl.gov/Caliper/>`_ | ||||||
| - `Hatchet User Guide <https://llnl-hatchet.readthedocs.io/en/latest/user_guide.html>`_ | ||||||
|
||||||
|
||||||
Caliper *annotation* is in the following tree structure | ||||||
|
||||||
Variant | ||||||
Group | ||||||
Kernel | ||||||
Kernel.Tuning | ||||||
|
||||||
| Build against these Caliper versions | ||||||
| | ||||||
| **caliper@2.9.0** (preferred target) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add checks for this in the top-level CMakeLists.txt file? These constraints should also be contained in the RAJAPerf Spack package. |
||||||
| **caliper@master** (if using older Spack version) | ||||||
|
||||||
In Cmake scripts add | ||||||
**-DRAJA_PERFSUITE_USE_CALIPER=On** | ||||||
|
||||||
Add to **-DCMAKE_PREFIX_PATH** | ||||||
;${CALIPER_PREFIX}/share/cmake/caliper;${ADIAK_PREFIX}/lib/cmake/adiak | ||||||
|
||||||
or use | ||||||
-Dcaliper_DIR -Dadiak_DIR package prefixes | ||||||
|
||||||
For Spack : raja_perf +caliper ^caliper@2.9.0 | ||||||
|
||||||
For Uberenv: python3 scripts/uberenv/uberenv.py --spec +caliper ^caliper@2.9.0 | ||||||
|
||||||
If you intend on passing nvtx or roctx annotation to Nvidia or AMD profiling tools, | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
build Caliper with +cuda cuda_arch=XX or +rocm respectively. Then you can specify | ||||||
an additional Caliper service for nvtx or roctx like so: roctx example: | ||||||
|
||||||
CALI_SERVICES_ENABLE=roctx rocprof --roctx-trace --hip-trace raja-perf.exe | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -159,3 +159,72 @@ storing the result in matrix A (N_i X N_j). Problem size could be chosen to be | |
the maximum number of entries in matrix B or C. We choose the size of matrix | ||
A (N_i * N_j), which is more closely aligned with the number of independent | ||
operations (i.e., the amount of parallel work) in the kernels. | ||
|
||
|
||
=========================== | ||
Caliper output files | ||
=========================== | ||
|
||
If you've built RAJAPerf with Caliper support turned on, then in addition to the | ||
outputs mentioned above, we also save a .cali file for each variant run, such as: | ||
Base_OpenMP.cali, Lambda_OpenMP.cali, RAJA_OpenMP.cali, etc. | ||
|
||
There are several techniques to display the Caliper trees (Timing Hierarchy) | ||
|
||
| 1: Caliper's cali-query tool. | ||
| The first technique is with Caliper's own tool cali-query, we run it with | ||
| **-T** to display tree, or you can specify **--tree**. | ||
| | ||
| cali-query -T $HOME/data/default_problem_size/gcc/RAJA_Seq.cali | ||
|
||
2: Caliper's Python module *caliperreader*:: | ||
|
||
import os | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this live in a file someplace? If so, we should include it here using comment delimiters in that file so that the docs here don't get out of sync with the script file. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, this is a sample script embedded in the docs. This and the one below I thought small enough to be included inline. I will revisit the small Hatchet script wrt divergence issues between the LC Hatchet, and PYPI Hatchet when Stephanie includes her updates. But, I see your point especially with my comment concerning version divergence. Maybe I can move it into the CI script as a free function, that can then be refer'ed to in the doc. What do you consider best practice wrt display snippets or gists in the documentation, where gists are maintained separately but get built into rst There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That would be good so it can be updated if needed and simplify doc maintenance. |
||
import caliperreader as cr | ||
DATA_DIR = os.getenv('HOME')+"/data/default_problem_size/gcc" | ||
os.chdir(DATA_DIR) | ||
r = cr.CaliperReader() | ||
r.read("RAJA_Seq.cali") | ||
metric = 'avg#inclusive#sum#time.duration' | ||
for rec in r.records: | ||
path = rec['path'] if 'path' in rec else 'UNKNOWN' | ||
time = rec[metric] if metric in rec else '0' | ||
if not 'UNKNOWN' in path: | ||
if (isinstance(path, list)): | ||
path = "/".join(path) | ||
print("{0}: {1}".format(path, time)) | ||
|
||
You can add a couple of lines to view the metadata keys captured by Caliper/Adiak:: | ||
|
||
for g in r.globals: | ||
print(g) | ||
|
||
You can also add a line to display metadata value in the dictionary **r.globals** | ||
|
||
For example print out the OpenMP Max Threads value recorded at runtime:: | ||
|
||
print('OMP Max Threads: ' + r.globals['omp_max_threads'])` | ||
|
||
or the variant represented in this file:: | ||
|
||
print('Variant: ' + r.globals['variant']) | ||
|
||
|
||
.. note:: The script above was written using caliper-reader 0.3.0, | ||
but is fairly generic. Other version usage notes may be | ||
found at the link below | ||
|
||
`caliper-reader <https://pypi.org/project/caliper-reader/>`_ | ||
|
||
|
||
3: Using the *Hatchet* Python module:: | ||
|
||
import hatchet as ht | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Similar comment as above about linking to section in the script file. |
||
DATA_DIR = os.getenv('HOME')+"/data/default_problem_size/gcc" | ||
os.chdir(DATA_DIR) | ||
gf1 = ht.GraphFrame.from_caliperreader("RAJA_Seq.cali") | ||
print(gf1.tree()) | ||
|
||
`Find out more on hatchet <https://github.com/LLNL/hatchet>`_ | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason to have a different USE_CALIPER for each project? Caliper regions+data are composable across projects, so I wonder if there is a use case where we would want to enable caliper in one project but not another - if not, keeping it the same across projects may be nice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One rationale for the different flag was the fact that Caliper support is not in BLT, where then it could be made general, including the way the flag propagates across subprojects.