-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core][Build] Move build module transformations and utilities to C++ #9103
Conversation
…plit action from mixed module
…mental_refactor
Thanks @mikepapadim . One thing that would be very helpful is to split the logic into two parts:
As part of python first principle, we want to enable ways to access and tweak optimizations in python easily. While majority of the default logic can sit in c++. It would be super helpful to have a python function that returns the composite pipeline, and possibly allow us to do further customizations (in python) if necessary and invoke them. |
Thanks @tqchen. So, we want to expose on the python side the ability to register additional passes for |
PTAL |
d8637a7
to
2162c85
Compare
@mbs-octoml @jroesch @tqchen this can be merged now |
return {host_mod, device_mod}; | ||
} | ||
|
||
runtime::Module FinalizeModule(const Map<Target, IRModule>& inputs_arg, const Target& host_target) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should probably come up with a better name for this function, finalize has too many technical meanings, probably better to call something which implies what is going on cc @mbs-octoml going to land as is, maybe we can make a note to follow up on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, IMO there is some more cleanup to be done in build_module
.
…pache#9103) * Initial investigation * More progress! * More progress / notes * rewrite build_for_device mostly in c++ * More progress * Initial split of transformations applied to device and host as post split action from mixed module * Combine duplicate passes after spliting mod on aot and vm flows * Minor cleanup * Move target mangling to driver_api.cc * Move more build utlities to cpp driver api * [Build][WIP] Moving build utilities to C++ from Python * [Build] Remove comments * [lint] Pass black * More formating * Move more build functionality into cpp * Remove comments * Remove unused defs and imports * Address PR comments * More PR comments * More comments * More comments * Add comments on the new split function * Fix PR comments on clarity * Test CI * Fix format * Refactor build * Expose splitted composite passes to python * Format files * Test fix * Fix for annotating entry funcs on code targeting CPU * Prevent entry funcs to be annotated when compiling for CPU with C runtime enabled * Guard for aot executor entry * Sphix format * Sanity fix * Sphinx fix Co-authored-by: electriclilies <lilyorthsmith@gmail.com>
…pache#9103) * Initial investigation * More progress! * More progress / notes * rewrite build_for_device mostly in c++ * More progress * Initial split of transformations applied to device and host as post split action from mixed module * Combine duplicate passes after spliting mod on aot and vm flows * Minor cleanup * Move target mangling to driver_api.cc * Move more build utlities to cpp driver api * [Build][WIP] Moving build utilities to C++ from Python * [Build] Remove comments * [lint] Pass black * More formating * Move more build functionality into cpp * Remove comments * Remove unused defs and imports * Address PR comments * More PR comments * More comments * More comments * Add comments on the new split function * Fix PR comments on clarity * Test CI * Fix format * Refactor build * Expose splitted composite passes to python * Format files * Test fix * Fix for annotating entry funcs on code targeting CPU * Prevent entry funcs to be annotated when compiling for CPU with C runtime enabled * Guard for aot executor entry * Sphix format * Sanity fix * Sphinx fix Co-authored-by: electriclilies <lilyorthsmith@gmail.com>
…pache#9103) * Initial investigation * More progress! * More progress / notes * rewrite build_for_device mostly in c++ * More progress * Initial split of transformations applied to device and host as post split action from mixed module * Combine duplicate passes after spliting mod on aot and vm flows * Minor cleanup * Move target mangling to driver_api.cc * Move more build utlities to cpp driver api * [Build][WIP] Moving build utilities to C++ from Python * [Build] Remove comments * [lint] Pass black * More formating * Move more build functionality into cpp * Remove comments * Remove unused defs and imports * Address PR comments * More PR comments * More comments * More comments * Add comments on the new split function * Fix PR comments on clarity * Test CI * Fix format * Refactor build * Expose splitted composite passes to python * Format files * Test fix * Fix for annotating entry funcs on code targeting CPU * Prevent entry funcs to be annotated when compiling for CPU with C runtime enabled * Guard for aot executor entry * Sphix format * Sanity fix * Sphinx fix Co-authored-by: electriclilies <lilyorthsmith@gmail.com>
This PR builds on top of @electriclilies initial work to move and unify build into C++
driver_api.cc
frombuild_module.py
.It removes redundant optimizations passes and introduces helpers functions for target mangling in the driver_api.
@mbs-octoml @jroesch @electriclilies @areusch