-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better kernel debug support #691
Conversation
nsmithtt
commented
Sep 14, 2024
•
edited
Loading
edited
- Introduce debug singleton object used to hold debug environment information during runtime. This could be a good place to store new kinds of debug flags in the future for runtime.
- Wire it up through ttrt.
- loadKernelsFromDisk added to denote reloading of generated kernels from /tmp, from previous run, instead of loading them from the flatbuffer.
- Name the kernels with program and location info from the MLIR graph. Now names look like:
- Usage with ttrt:
38339d3
to
f9a1a2c
Compare
- Introduce debug singleton object used to hold debug environment information during runtime. This could be a good place to store new kinds of debug flags in the future for runtime. - Wire it up through ttrt. - loadKernelsFromDisk added to denote reloading of generated kernels from /tmp, from previous run, instead of loading them from the flatbuffer. - Name the kernels with program and location info from the MLIR graph. Now names look like /tmp/ttmlir_multiply_%5_tensix.cpp ^ ^ ^ Func name -/ | | | | Result Value Loc -/ | | Thread type/info [noc, eth, tensix] -/
f9a1a2c
to
9e6f47c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks awesome. Was just realizing today that program name wasn't visible to kernels so this is timely and handy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!
- Introduce debug singleton object used to hold debug environment information during runtime. This could be a good place to store new kinds of debug flags in the future for runtime. - Wire it up through ttrt. - loadKernelsFromDisk added to denote reloading of generated kernels from /tmp, from previous run, instead of loading them from the flatbuffer. - Name the kernels with program and location info from the MLIR graph. Now names look like /tmp/ttmlir_multiply_%5_tensix.cpp ^ ^ ^ Func name -/ | | | | Result Value Loc -/ | | Thread type/info [noc, eth, tensix] -/
- Introduce debug singleton object used to hold debug environment information during runtime. This could be a good place to store new kinds of debug flags in the future for runtime. - Wire it up through ttrt. - loadKernelsFromDisk added to denote reloading of generated kernels from /tmp, from previous run, instead of loading them from the flatbuffer. - Name the kernels with program and location info from the MLIR graph. Now names look like /tmp/ttmlir_multiply_%5_tensix.cpp ^ ^ ^ Func name -/ | | | | Result Value Loc -/ | | Thread type/info [noc, eth, tensix] -/
@nsmithtt I noticed we dump only 1 kernel per thread type, but we create data movement kernels for every core independently. And maybe we'll have split compute kernels per core groups too. Should we dump all of them? |
Good point, we should encode the core grid shape / offset in the string name, i.e. one kernel per entry in the |
Sure, #794 issue tracks it, will fix it at some point. |