[r2.3] backport: auto-sharding PoC (#6719)

[yeounoh]

backport PR#6719

[yeounoh]

@JackCaoG the build is blocked on this change, that is not backported already,

torch_xla/csrc/xla_graph_executor.cpp: In member function ‘torch::lazy::LazyGraphExecutor::SyncTensorCollection torch_xla::XLAGraphExecutor::CollectSyncTensors(const std::vector<c10::intrusive_ptr<torch_xla::XLATensor> >&, const torch::lazy::LazyGraphExecutor::SyncTensorsConfig&)’:
torch_xla/csrc/xla_graph_executor.cpp:638:62: error: ‘using element_type = struct torch_xla::XLATensor::Data’ ***aka ‘struct torch_xla::XLATensor::Data’*** has no member named ‘is_cloned’
  638 |           if (device_data != nullptr && !tensors[i]->data()->is_cloned) ***
      |                                                              ^~~~~~~~~
INFO: Elapsed time: 320.302s, Critical Path: 59.47s
INFO: 30772 processes: 15751 remote cache hit, 14836 internal, 185 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
INFO: Streaming build results to: https://source.cloud.google.com/results/invocations/46e20649-0253-4ec2-9207-1d1220dc8bb7
/opt/conda/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.

This is the new change you landed. So should we just revert this part to prior to your change or we are going to back port the conflicting change, too?

[JackCaoG]

it is safer to revert related part of my change. My pr was a bit dangerous so I want to leave it in nightly for now.