Add initial LTC backend

[antoniojkim]

Adding the initial Torch-MLIR LTC backend. There are still many unimplemented components, but this PR provides the general skeleton for the backend API.

A CMakeLists.txt was included as well to enable the building of the LTC backend. It does currently build. However, as mentioned above, there are still many unimplemented components. So, it will not run.

As per the README:

Contained within this directory are the components that implements the
Torch-MLIR LTC backend.

The components are subclasses of the backend API interface classes found under
torch/csrc/lazy/backend.

Importantly, the subclasses are still abstract classes. Pure virtual methods
such as Compile were purposefully not overriden as Torch-MLIR does not know
how to compile the model for the target hardware.

The intent is that vendor hardware specific plugins will subclass the Torch-MLIR
backend classes and override the remaining pure virtual functions to complete
the backend.

The Torch-MLIR LTC backend's job is to perform the lowering from ATen to MLIR. A
hardware vendor's backend job is to take care of the actual compile and
execution of the lowered MLIR.

CC: @henrytwo @ke1337

[silvasean]

Thanks for getting the ball rolling on this. Let's do this!

[silvasean]

@antoniojkim - can you comment on when would be a good time to intercept this to implement an IREE backend here? I just want to stay on top of that so we can start work on that as soon as possible!

[antoniojkim]

torch-mlir/python/torch_mlir/csrc/backend/mlir_node.h:4:10: fatal error: 'torch/csrc/lazy/backend/lowering_context.h' file not found
#include <torch/csrc/lazy/backend/lowering_context.h>

Hmm, okay. The build is failing because there isn't a torch installation found. @silvasean How is this done normally for Torch-MLIR? I know that another module requires torch as well. I think a nightly cpu build will suffice here.

[antoniojkim]

@antoniojkim - can you comment on when would be a good time to intercept this to implement an IREE backend here? I just want to stay on top of that so we can start work on that as soon as possible!

I think a good time might be when we have a minimum viable product. Currently, there is a lot of unimplemented functionality here. Once we have something working I think we can start adding builds and tests for it.

What I'm planning to do is to create an example vendor backend that subclasses the Torch-MLIR backend and include that to use for testing purposes. I'm thinking that it would also serve as an example for how to create a custom vendor backend that makes use of Torch-MLIR. Thoughts?

[silvasean]

You can see how we do it in the existing JIT IR importer here:

torch-mlir/python/torch_mlir/dialects/torch/importer/jit_ir/CMakeLists.txt

Line 5 in ed9bd55

list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules")

From a bigger picture perspective, we probably shouldn't have the PR CI running on this branch. I think you should be able to configure that by adding something here:

torch-mlir/.github/workflows/buildAndTest.yml

Line 7 in ed9bd55

pull_request:

I think something like this: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#running-your-workflow-based-on-the-head-or-base-branch-of-a-pull-request

  pull_request:
    branches:    
      - 'main'

[silvasean]

What I'm planning to do is to create an example vendor backend that subclasses the Torch-MLIR backend and include that to use for testing purposes. I'm thinking that it would also serve as an example for how to create a custom vendor backend that makes use of Torch-MLIR. Thoughts?

That seems right to me! Would the example vendor backend be able to actually run anything / how would it do it? If we can run some basic stuff without IREE that would be very useful.

[antoniojkim]

That seems right to me! Would the example vendor backend be able to actually run anything / how would it do it? If we can run some basic stuff without IREE that would be very useful.

I don't think it'll be able to run anything useful as that's entirely vendor hardware dependant. But I think we can have some tests that verify that we were able to get proper ATen -> MLIR lowering via the Torch-MLIR backend supplemented with the example vendor backend

The big thing to test here I think is the lowering. Secondary to that, we can test that the example vendor's Compile and Execute are receiving properly lowered MLIR to be dealt with however the vendor wants.

[silvasean]

Ok, cool!