[feat] Sync OSS.param_groups and the sharded optimizer param_groups

[blefaudeux]

🚀 Feature

Make sure that all attributes (not just LR) are in sync in between the OSS.param_groups and the actual optimizer.

Motivation

Some frameworks make it possible to alter any attribute here, not just LR (momentum, beta for Adam, ..). We do not currently support this and silently fail

Pitch

Part of the not really well defined PyTorch optimizer features which are nice to have

Alternatives

• At least add a warning when we're out of sync
• Nuke the .param_groups access
• Force a new OSS optimizer to be created everytime the user wants to change an attribute in param_groups

Additional context