Make custom ops compatible with torch.compile

[akihironitta]

🚀 The feature, motivation and pitch

The PyTorch team has recently released a guide [1] to making custom C++ ops compatible with torch.compile. Although the API is still at an early stage, we can start exploring whether we can make custom ops in PyG's optional libraries, such as pyg-lib, and giving feedbacks to PyTorch as necessary.

Currently, we handle the imcompatibility issue by falling back to (slow) equivalent operations written in pure PyTorch where possible (like this previously or like this in PyG 2.5.0+)

Refs:

1. [GUIDE] Getting C++ custom ops to work with torch.compile - PyTorch Dev Discussion
2. The C++ Custom Operators Manual - Google Docs
3. The dynamic shapes manual - Google Docs
4. [dynamo] Investigate interop issues with torch_scatter/torch_sparse/pyg_lib pytorch/pytorch#111223

Alternatives

No response

Additional context

No response