Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[M3c][Meta Schedule] Implement the Replay Func class. #495

Merged
merged 9 commits into from
Oct 28, 2021
Merged

[M3c][Meta Schedule] Implement the Replay Func class. #495

merged 9 commits into from
Oct 28, 2021

Conversation

zxybazh
Copy link
Collaborator

@zxybazh zxybazh commented Oct 27, 2021

Add implementation of the replay func class. Test included.
The replay func class can invoke a space generator to modify the given design spaces.

Note that:

  1. The given design spaces are fed into the space generator via Pretuning where the design spaces, i.e, schedules, are random selected and transformed into an IRModule. And we store the IRModules in an array to avoid threading issues.
  2. Each time one schedule is randomly selected as the output.

@zxybazh zxybazh changed the title Implement the Replay Func class. [M3c][Meta Schedule]Implement the Replay Func class. Oct 27, 2021
@zxybazh zxybazh changed the title [M3c][Meta Schedule]Implement the Replay Func class. [M3c][Meta Schedule] Implement the Replay Func class. Oct 27, 2021
@zxybazh
Copy link
Collaborator Author

zxybazh commented Oct 28, 2021

Modified the implementation as suggested:

  1. The space generator is directly fetched from the tune context when doing initialization with tune context.
  2. The IRModule to process is fetched directly from tune context when using the space generator in search strategy,
  3. Multi-thread is removed, tested with cython enabled.

Thanks @junrushao1994 for the careful review.

@junrushao
Copy link
Member

@zxybazh this is definitely very interesting finding that ctypes ffi doesn't lead to python deadlock! If the deadlock only happens in cython, it means that we only need to update env func registration logic in cython like what's done in apache/tvm#7919

junrushao
junrushao approved these changes Oct 28, 2021
View reviewed changes
Copy link
Member

@junrushao junrushao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Let's get this PR in after the nit fix

@zxybazh
Copy link
Collaborator Author

zxybazh commented Oct 28, 2021

Just double checked, the PR is in good shape.

@zxybazh zxybazh merged commit ca3f72c into tlc-pack:meta-schedule-refactor Oct 28, 2021
junrushao pushed a commit that referenced this pull request Nov 5, 2021
@zxybazh @junrushao
[M3c][Meta Schedule] Implement the Replay Func class. (#495)
dcd4cd7 
* Add replay func.

* Modify pretuning.

* Modify docs.

* Minor nit.

* Enhance unittest.

* Nit.

* Fix replay func.

* Simplify unittest.

* Fix PyClass reflection.
junrushao added a commit that referenced this pull request Nov 5, 2021
@zxybazh @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @junrushao @vinx13 @sunggg
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)
f6330ae 
* Add schedule rule c++ side.

* Add postproc c++ side.

* Add mutator c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Fix printer and add test for schedule rule.

* Add test for mutator.

* Finish postproc tests.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add to tune context.

* Fix nits & test bug.

[Meta Schedule][M3c] PostOrderApply (#486)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Add python-side PostOrderApply with test.

* Fix test.

* Fix warning.

* Fix TaskScheduler Init Rules.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Move numerate order.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Fix Pretuning location in TaskScheduler.

* Fix and refactor.

* Fix test.

* Fix unit test.

Fix Post Order Apply (#490)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Fix post order.

* Fix blocks.

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Add trace  correctness test for post order apply.

Fix replay trace. (#493)

* Fix replay trace.

* Fix deep copy.

* Fix init & add docs.

* Add back function.

* Nit.

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

* Add replay func.

* Modify pretuning.

* Modify docs.

* Minor nit.

* Enhance unittest.

* Nit.

* Fix replay func.

* Simplify unittest.

* Fix PyClass reflection.

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

* [PR] Test script for meta-schedule task extraction. Interface to load workloads from Torch zoo is also added.

* Replace relay_workload to torch_workload. Add 50+ models.

[Meta Schedule Refactor] Get child blocks (#500)

* get child blocks

* clang format

* black

* test

* test

* test

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

* Add MeasureCallbacks.

* ove measure callback to task scheduler level.

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

* Add replay func.

* Simplify unittest.

* Fix the infinite loop for not implemented methods in PyClass.

* Fix funcname.

* Restore the test.

* Fix all the PyClaes with optional function override.

* Fix __str__ and __len__.

* Move NotImplementedError to declaration.

* Add docs.

* Rebase.

[MetaSchedule] Sample-Perfect-Tile (#501)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Nov 5, 2021
@zxybazh @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @junrushao @vinx13 @sunggg
Squashed commit
aacb13b 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add mutator c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Fix printer and add test for schedule rule.

* Add test for mutator.

* Finish postproc tests.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add to tune context.

* Fix nits & test bug.

[Meta Schedule][M3c] PostOrderApply (#486)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Add python-side PostOrderApply with test.

* Fix test.

* Fix warning.

* Fix TaskScheduler Init Rules.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Move numerate order.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Fix Pretuning location in TaskScheduler.

* Fix and refactor.

* Fix test.

* Fix unit test.

Fix Post Order Apply (#490)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Fix post order.

* Fix blocks.

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

* Add schedule rule c++ side.

* Add postproc c++ side.

* Add schedule rule python side.

* Add mutator python side.

* Add Postproc python side.

* Put src to different files in python.

* Try to fix bprint.

* Get str function work.

* Add PostOrderApply c++ side.

* Remove IRModule from SpaceGenerator's generate function signature.

* Refactor PostOrderApply.

* Fix PostOrderApply Design.

* Change StmtSRef to BlockRV.

* Add IRModule back to SpaceGenerator.

* Add trace  correctness test for post order apply.

Fix replay trace. (#493)

* Fix replay trace.

* Fix deep copy.

* Fix init & add docs.

* Add back function.

* Nit.

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

* Add replay func.

* Modify pretuning.

* Modify docs.

* Minor nit.

* Enhance unittest.

* Nit.

* Fix replay func.

* Simplify unittest.

* Fix PyClass reflection.

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

* [PR] Test script for meta-schedule task extraction. Interface to load workloads from Torch zoo is also added.

* Replace relay_workload to torch_workload. Add 50+ models.

[Meta Schedule Refactor] Get child blocks (#500)

* get child blocks

* clang format

* black

* test

* test

* test

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

* Add MeasureCallbacks.

* ove measure callback to task scheduler level.

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

* Add replay func.

* Simplify unittest.

* Fix the infinite loop for not implemented methods in PyClass.

* Fix funcname.

* Restore the test.

* Fix all the PyClaes with optional function override.

* Fix __str__ and __len__.

* Move NotImplementedError to declaration.

* Add docs.

* Rebase.

[MetaSchedule] Sample-Perfect-Tile (#501)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Nov 5, 2021
@zxybazh @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @junrushao @vinx13 @sunggg
Squashed commit
a2b4a94 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Nov 5, 2021
@zxybazh @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @junrushao @vinx13 @sunggg
Squashed commit
46919f7 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Dec 1, 2021
@zxybazh @junrushao
[Meta Schedule] Add XGBoost Model & Random Model (#519)
1c35161 
* Squashed commit

[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

* annotate

* annotate

* lint

* test

* fix

* fix

* fix

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

Fix sttr func & schedule naming.

Fix schedule -> sch.

Add feature extractor.

Fix init.

Add cost model.

Remove unused include.

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

* wip

fix

* revoke change to gallery

* split postprocessors to separate files

* rename attrs

* minor

* minor tweak on utils.h

* refactor disallow-dynamic-loop

* refactor verify_gpu_code

* succesfully give up refactoring parallelize-vectorize-unroll

* python structuring

* unittests

Co-authored-by: Junru Shao <junrushao1994@gmail.com>

Fix issues.

Fix init.

Finish random model part.

Finish xgb model.

Minor fix.

Rebase.

Add init.

Await refactor of callback.

Update a bit on the test case.

Move impos.

Minor fix.

More fixes.

Remove unused import.

Fix per store feature test.

Update model save / load.

* Fix model save / load with tar.

* Fix issues.

* Remove dup.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
junrushao added a commit that referenced this pull request Dec 7, 2021
@zxybazh @junrushao
@Hzfengsy @spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg
[Meta Schedule] Evolutionary Search (#522)
1249af5 
* Checkpoint.

Fix cost model comment.

Finish evolutionary seaarch.

Remove  extra code.

Fix compile.

Add comments.

Add python part.

Ad test.

Update other files & comments.

* Squashed commit

[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

* [TensorIR] GetProducer, GetConsumer (#506)

* [MetaScheduleRefactor] Annotate&Unannotate (#505)

* annotate

* annotate

* lint

* test

* fix

* fix

* fix

* [MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

* Blockize & Tensorize (#514)

* Blockize & Tensorize

* Update tensor intrin

* Fix blockized & Recalculate affine flags

* Cleanup utils.cc

* Add test cases of blockize

* Re-enable affine flag checking

* Checkpoint.

Fix cost model comment.

Finish evolutionary seaarch.

Remove  extra code.

Fix compile.

Add comments.

Add python part.

Ad test.

Update other files & comments.

Fix random seed bug.

Minor fix.

Fix num-cores.

Add docs.

Check point.

Add max_fail_cnt.

Minor fix.

Minor fix.

Segfault.

Fix pointers to trace.

Test fix.

Remove measure callbacks.

Refactor a bit.

Split function.

Adjust variable name.

Minor fixes.

Add mutator probs to TuneContext.

Add token.

Fix loops.

Remove include.

Add has workload for database.

Add check.

Add concurrent bitmask.

* Fix TuneContext.

* Fix haash & stuff.

* Modifyy shash.

* Remove trace field.

* Minor fix.

* Fix cbmask.

* Fix numbers.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Dec 7, 2021
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg
Squashed commit
539e86d 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (#508)

[Meta Schedule] Minor Fixes (#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

[Meta Schedule] Add Helper Function & Minor Modification (#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (#513)

[Meta Schedule] Feature Extractor & Cost Model (#510)

Blockize & Tensorize (#514)

Layout Rewriting: Suggest-Index-Map (#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (#516)

[Meta Schedule] Per-Store-Feature (#521)

Add traced schedule for blockize & tensorize (#526)

[Meta Schedule] Add XGBoost Model & Random Model (#519)

User-Interface: Tune-TIR (#525)

User-Interface: Tune-TE (#527)

[Minor] More logging on python (#528)

Get CUDA tuning working (#529)

[MetaSchedule] TensorRT BYOC (#518)

[BugFix] LocalBuilder API (#531)

[Meta Schedule] Add Cost Model Update Measure Callback (#530)

[Bugfix] BuilderInput with default params (#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (#534)

[Meta Schedule] Evolutionary Search (#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (#535)

[Meta Schedule] Fix some bugs (#537)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Dec 7, 2021
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg
Squashed commit
6c6489e 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (#508)

[Meta Schedule] Minor Fixes (#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

[Meta Schedule] Add Helper Function & Minor Modification (#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (#513)

[Meta Schedule] Feature Extractor & Cost Model (#510)

Blockize & Tensorize (#514)

Layout Rewriting: Suggest-Index-Map (#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (#516)

[Meta Schedule] Per-Store-Feature (#521)

Add traced schedule for blockize & tensorize (#526)

[Meta Schedule] Add XGBoost Model & Random Model (#519)

User-Interface: Tune-TIR (#525)

User-Interface: Tune-TE (#527)

[Minor] More logging on python (#528)

Get CUDA tuning working (#529)

[MetaSchedule] TensorRT BYOC (#518)

[BugFix] LocalBuilder API (#531)

[Meta Schedule] Add Cost Model Update Measure Callback (#530)

[Bugfix] BuilderInput with default params (#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (#534)

[Meta Schedule] Evolutionary Search (#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (#535)

[Meta Schedule] Fix some bugs (#537)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
junrushao added a commit that referenced this pull request Dec 7, 2021
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg @zxybazh
Squashed commit
7ae0d98 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (#508)

[Meta Schedule] Minor Fixes (#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

[Meta Schedule] Add Helper Function & Minor Modification (#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (#513)

[Meta Schedule] Feature Extractor & Cost Model (#510)

Blockize & Tensorize (#514)

Layout Rewriting: Suggest-Index-Map (#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (#516)

[Meta Schedule] Per-Store-Feature (#521)

Add traced schedule for blockize & tensorize (#526)

[Meta Schedule] Add XGBoost Model & Random Model (#519)

User-Interface: Tune-TIR (#525)

User-Interface: Tune-TE (#527)

[Minor] More logging on python (#528)

Get CUDA tuning working (#529)

[MetaSchedule] TensorRT BYOC (#518)

[BugFix] LocalBuilder API (#531)

[Meta Schedule] Add Cost Model Update Measure Callback (#530)

[Bugfix] BuilderInput with default params (#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (#534)

[Meta Schedule] Evolutionary Search (#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (#535)

[Meta Schedule] Fix some bugs (#537)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
zxybazh added a commit that referenced this pull request Dec 10, 2021
@zxybazh @junrushao
@Hzfengsy @spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg
[Meta Schedule] Add Replay Tuning Interface (#543)
7eed2b0 
* Squashed commit

[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (#508)

[Meta Schedule] Minor Fixes (#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

[Meta Schedule] Add Helper Function & Minor Modification (#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (#513)

[Meta Schedule] Feature Extractor & Cost Model (#510)

Blockize & Tensorize (#514)

Layout Rewriting: Suggest-Index-Map (#520)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

* [MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (#516)

* parallel vectorize unroll & random compute location

* rebased

* [Meta Schedule] Per-Store-Feature (#521)

* [Meta Schedule] Add XGBoost Model & Random Model (#519)

* Squashed commit

[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

* annotate

* annotate

* lint

* test

* fix

* fix

* fix

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

Fix sttr func & schedule naming.

Fix schedule -> sch.

Add feature extractor.

Fix init.

Add cost model.

Remove unused include.

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

* wip

fix

* revoke change to gallery

* split postprocessors to separate files

* rename attrs

* minor

* minor tweak on utils.h

* refactor disallow-dynamic-loop

* refactor verify_gpu_code

* succesfully give up refactoring parallelize-vectorize-unroll

* python structuring

* unittests

Co-authored-by: Junru Shao <junrushao1994@gmail.com>

Fix issues.

Fix init.

Finish random model part.

Finish xgb model.

Minor fix.

Rebase.

Add init.

Await refactor of callback.

Update a bit on the test case.

Move impos.

Minor fix.

More fixes.

Remove unused import.

Fix per store feature test.

Update model save / load.

* Fix model save / load with tar.

* Fix issues.

* Remove dup.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>

* User-Interface: Tune-TIR (#525)

* User-Interface: Tune-TIR

* fix fix fix

* User-Interface: Tune-TE (#527)

* fix a lot of issues

* Add tune-te

* Get CUDA tuning working (#529)

* [Meta Schedule] Evolutionary Search (#522)

* Checkpoint.

Fix cost model comment.

Finish evolutionary seaarch.

Remove  extra code.

Fix compile.

Add comments.

Add python part.

Ad test.

Update other files & comments.

* Squashed commit

[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

* [TensorIR] GetProducer, GetConsumer (#506)

* [MetaScheduleRefactor] Annotate&Unannotate (#505)

* annotate

* annotate

* lint

* test

* fix

* fix

* fix

* [MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

* Blockize & Tensorize (#514)

* Blockize & Tensorize

* Update tensor intrin

* Fix blockized & Recalculate affine flags

* Cleanup utils.cc

* Add test cases of blockize

* Re-enable affine flag checking

* Checkpoint.

Fix cost model comment.

Finish evolutionary seaarch.

Remove  extra code.

Fix compile.

Add comments.

Add python part.

Ad test.

Update other files & comments.

Fix random seed bug.

Minor fix.

Fix num-cores.

Add docs.

Check point.

Add max_fail_cnt.

Minor fix.

Minor fix.

Segfault.

Fix pointers to trace.

Test fix.

Remove measure callbacks.

Refactor a bit.

Split function.

Adjust variable name.

Minor fixes.

Add mutator probs to TuneContext.

Add token.

Fix loops.

Remove include.

Add has workload for database.

Add check.

Add concurrent bitmask.

* Fix TuneContext.

* Fix haash & stuff.

* Modifyy shash.

* Remove trace field.

* Minor fix.

* Fix cbmask.

* Fix numbers.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>

* [BugFix] Remove duplicated definition of MakeMultinomialSampler (#535)

* Tune relay.

* [FIX] Simplify during create prim func (#9691)

* Further add interface.

* Remove unused import

* Fix rebase.

* Renmove unused module.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
spectrometerHBH added a commit that referenced this pull request Dec 30, 2021
@Hzfengsy @spectrometerHBH
@jinhongyii @MasterJH5574 @junrushao @vinx13 @sunggg @zxybazh
Squashed commit
ae4b33d 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (#485)

[Meta Schedule][M3c] PostOrderApply (#486)

Fix Post Order Apply (#490)

[MetaSchedule] Relay Integration (#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (#492)

Fix replay trace. (#493)

[M3c][Meta Schedule] Implement the Replay Func class. (#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (#494)

[Meta Schedule Refactor] Get child blocks (#500)

Read-at && Write-at (#497)

[M3c][Meta Schedule] Measure Callbacks (#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (#496)

[MetaSchedule] Sample-Perfect-Tile (#501)

[MetaSchedule] TE Workloads (#502)

[TensorIR] GetProducer, GetConsumer (#506)

[MetaScheduleRefactor] Annotate&Unannotate (#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (#508)

[Meta Schedule] Minor Fixes (#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (#499)

[Meta Schedule] Add Helper Function & Minor Modification (#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (#513)

[Meta Schedule] Feature Extractor & Cost Model (#510)

Blockize & Tensorize (#514)

Layout Rewriting: Suggest-Index-Map (#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (#516)

[Meta Schedule] Per-Store-Feature (#521)

Add traced schedule for blockize & tensorize (#526)

[Meta Schedule] Add XGBoost Model & Random Model (#519)

User-Interface: Tune-TIR (#525)

User-Interface: Tune-TE (#527)

[Minor] More logging on python (#528)

Get CUDA tuning working (#529)

[MetaSchedule] TensorRT BYOC (#518)

[BugFix] LocalBuilder API (#531)

[Meta Schedule] Add Cost Model Update Measure Callback (#530)

[Bugfix] BuilderInput with default params (#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (#534)

[Meta Schedule] Evolutionary Search (#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (#535)

[Meta Schedule] Fix some bugs (#537)

Initiate Experiments for CPU Performance Alignment with Ansor (#538)

[Meta Schedule] Tweak experiment scripts (#539)

[Meta Schedule] Initiate experiments on CUDA (#540)

[TIR][Schedule] Buffer transform (#523)

Auto Tensor Core (#524)

Working on Evo Search (#542)

[Meta Schedule] Add Replay Tuning Interface (#543)

Evolutionary Search on CPU (#544)

Misc improvement over the error message (#545)

[TIR][Schedule] Software pipelining (#533)

[Meta Schedule Refactor] fixing unit tests (#547)

[MetaSchedule] Mutator-Compute-Location (#548)

Misc Improvement of Evolutionary Search (#549)

Hotfix for software pipeline (#552)

Misc Improvement (#550)

[Cherry-Pick][TensorIR] Primitive "SetScope" (#9738) (#555)

Rule RFactor (#551)

[MemHammer] Rewrite Rules (#554)

[MetaSchedule] Schedule Rule: Cross-Thread Reduction (#556)

[MetaSchedule] Performance Alignment - NRM and SFM (CUDA) (#559)

[MetaSchedule] Perf Alignment - NRM on CUDA (#560)

[TIR] Reorder the block iters of the blocks generated by RFactor (#561)
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@junrushao junrushao junrushao approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zxybazh @junrushao