[TorchDISC] compile disc nodes with a fake cluster algorithm #173
Conversation
Yancey1989
changed the title
Yancey1989
changed the title
| const std::set<int8_t> TSBackendDeviceType::supported_device_types_ = { | ||
| (int8_t)at::kCPU, (int8_t)at::kCUDA}; | ||
|
|
||
| class DISCBackendImpl : public torch::lazy::BackendImplInterface { |
There was a problem hiding this comment.
I think we should discuss this with PyTorch LTC. We could propose our roadmap and our requirements on LTC?
There was a problem hiding this comment.
Good point, maybe we should write a detailed design document before communicating with LTC team.
| // 2. conversion from torchscript Graph to mhlo Dialect on DISC nodes | ||
| // 2. register DISC engine | ||
| InferShapes(graph, arguments); | ||
| FusionDiscNodes(graph); |
There was a problem hiding this comment.
We'd better avoid to use "fusion" here, which can easily be misunderstood
There was a problem hiding this comment.
Good point. Actually, this function just CLUSTER instead of FUSION, maybe better to ClusterDiscNodes ?
| // 1. clustering and group DISC nodes | ||
| // 2. conversion from torchscript Graph to mhlo Dialect on DISC nodes | ||
| // 2. register DISC engine | ||
| InferShapes(graph, arguments); |
There was a problem hiding this comment.
I guess we are involving the lazy arguments for dtype and rank analysis, am i understanding it correct?
Without an explicit "static_rank_backward_analysis" process, it should be not enough for us to guarantee that all the compile-time-constants for static rank to be known at compile time. To be discussed offline.
compile Disc nodes with a fake cluster algorithm.
…#173) compile Disc nodes with a fake cluster algorithm.
This PR implements the compilationo stage with a fake cluster algorithm.