increase default ppo batch size from 64 to 256

[matwilso]

PPO can work poorly with small batch sizes, so 256 is probably a better default for people than 64. Also, 256 is the old default batch size in the original stable baselines (#90 (comment)), as well SAC currently.

I thought this was a minor enough change not to require the normal PR process. But let me know if you want me to go through that.

[Miffyli]

Hello. Please open up an issue discussing this change. The "non-issue PRs" are only reserved for hotfixes and very small updates like documentation typo fixes. This would change the behaviour of existing codebases, so it needs to be properly discussed.

[araffin]

Hello,

I'm not sure where you have seen that SB2 PPO had 256 as default batch size (the comment you linked was only valid for a particular example).
Make sure to read our migration guide: https://stable-baselines3.readthedocs.io/en/master/guide/migration.html
and afterward open an issue (as mentioned by @Miffyli ) if you think there is still one.

PS: please read our contributing guide and keep the PR template next time.