[RLlib; docs] Docs do-over (new API stack): RLModules vol 02; Major cleanups, SAC/DQN default RLModules, QNetAPI, remove noisy nets from DQN.

[sven1977]

Docs do-over (new API stack): RLModules vol 02

• Move all algo-specific RLModules into the "DefaultRLModule" pattern. Users don't have to subclass these default RLModules anymore as superclasses, but can write arbitrary custom RLModule as long as the custom RLModule implements the algo-required RLModule APIs. E.g. for training w/ PPO, the user's RLModule must implement the ValueFunctionAPI, BUT does not have to subclass anymore from PPORLModule!
• Major cleanups as a consequence of the above change in example scripts and elsewhere.
• Add new QNetAPI (for both DQN- and SACRLModules), requiring the user to implement the compute_q_values and (optionally) compute_advantage_distribution methods (for dueling-Q).
• Remove all code related to noisy nets in DQN. Having this exploration method was somewhat a relic from the old API stack's ParameterNoise Exploration component.

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[simonsays1980]

LGTM. Great PR. As we do not name DQN anymore RainbowDQN gthe removal of NoisyNets looks good.

[simonsays1980]

Awesome! So much cleanup!

[simonsays1980]

Great! This API was missing, yet.