[RLlib; Offline RL] - Replace GAE in MARWILOfflinePreLearner with GeneralAdvantageEstimation connector in learner pipeline.

[simonsays1980]

Why are these changes needed?

In PPO the GAE in the Learner had been replaced by the GeneralAdvantageEstimation connector in its learner pipeline. MARWIL basically needs the same calculation albeit not directly in the learner but in its PreLearner. This PR proposes this change.

• It adds the AddOneTsToEpisodesAndTruncate and GeneralAdvantageEstimation to MARWIL's learner pipeline.
• Deprecates the MARWILOfflinePreLearner b/c with the modified learner pipeline 'MARWIL' can fall back onto the default OfflinePreLearner.
• Changed the default for episodes in offline batches from finalize=False to finalize=True b/c this is needed for the AddOneTsToEpisodesAndTruncate connector.

Related issue number

Related to #47401

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[sven1977]

Approved! Nice consolidation/cleanup PR. Good stuff @simonsays1980 !

[sven1977]

One nit nit (see comments).