PoC: Accelerator refactor [wip] [skip ci]

[justusschock]

What does this PR do?

This PR is a revamp of #5385 since I messed up with git.
Closes #5385

Fixes #4510

This PR separates the Accelerator (Hardware Part) from the actual different training routines.

Workflow actions:

• Move current accelerators to legacy - change imports (still use them for training) >> Refactor: 1/n legacy accelerators and plugins  #5645
• Move current accelerator connector to legacy (still use it for training) and add new connector (necessary for tests)
• Add new accelerators (still use legacy for training)
• Add new plugins one by one (still use legacy for training)
• Add missing logic
• Drop Legacy for new backends. (Not sure if keeping them for next release).

Remaining TODOs:

• TPUAccelerator (Justus)
• DDP2 Plugin (Adrian)
• Shared Training Plugin (Sean/Justus)
• RPC Plugin (Justus)
• Add Layers for parallel: SingleProcessMultipleDevice (SPMD), SingleProcessSingleDevice (SPSD) (Justus)
• Rebase on release branch
• Testing DDP2 and DDP Slurm
• Remove old plugins (pl/plugins/old)
• Make Tuner work (requires setting some attrs through trainer) (Adrian)
• Port the left-over functions (like block_backward_sync and prepare_for_backward) from old DDPPlugin to new to avoid performance hits (Adrian/Justus)

So far this PR was co-authored with @awaelchli !

cc @awaelchli @tchaton @SeanNaren who will likely work on this!

Slides for motivation and high-level overview

List of PRs to look out for (when rebasing, code we need to manually copy over to new files):
#5221, #5195, #5300, #5388

[Borda]

@justusschock took me almost two hours to rebase and resolve conflicts on the way, but still, can you check that all is fine?

[awaelchli]

@Borda Thank you.
I added some commits to make the tests run again.
What is missing is the device_type and distrib_type refactor changes that you recently made. Could you help integrate them?
I can also assist

[Borda]

What is missing is the device_type and distrib_type refactor changes that you recently made. Could you help integrate them?
I can also assist

Happy to help =)