Support hierarchical dict

[S-aiueo32]

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?
• If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #1144.

The main change of this PR is adding the function _flatten_dict to LightningLoggerBase.
This function flattens hierarchical dictionaries and nested Namespace objects:

# hierarchical dict
_flatten_dict({'a': {'b': 'c'}})  # => {'a/b': 'c'}

# nested Namespace
namespace = Namespace(foo=Namespace(bar='buzz'))
namespace = vars(namespace)  # this conversion for top-level is done in `_convert_params`
_flatten_dict(namespace )  # => {'foo/bar': 'buzz'}

Considering the compatibility to all loggers, _flatten_dict is called in the log_hyperparams function of each logger class that needs flattening:

def log_hyperparams(self, params: Union[Dict[str, Any], Namespace]) -> None:
    params = self._convert_params(params)
    params = self._flatten_dict(params)

In other words, this function is not called by the loggers as represented by WanbLogger, which supports hierarchical dictionaries by default, to keep them the hierarchy.

Note1: WanbLogger supports hierarchical dictionaries, but not nested Namespace. It does not raise an error, however, the output yaml file on the UI is displayed as:

foo:
  desc: null
  value: Namespace(bar='buzz')

Note2: TRAINS also supports hierarchical dictionaries, but the UI destroys the hierarchy as done by _flatten_dict. To add support for nested Namespace, TrainsLogger calls _flatten_dict.

[codecov]

Codecov Report

Merging #1152 into master will increase coverage by <1%.
The diff coverage is 95%.

@@           Coverage Diff           @@
##           master   #1152    +/-   ##
=======================================
+ Coverage      93%     93%   +<1%     
=======================================
  Files          61      61            
  Lines        2672    2691    +19     
=======================================
+ Hits         2472    2491    +19     
  Misses        200     200

[awaelchli]

very nice!

[Borda]

LGTM 🚀

[KeAWang]

Is there a reason that wandb.py wasn't changed to flatten dict?

[awaelchli]

probably got forgotten, is already done, or not necessary. Did you check, is it necessary to do it @KeAWang ?

[awaelchli]

See here: https://docs.wandb.com/library/config

You can send us a nested dictionary in config, and we'll flatten the names using dots in our backend.

They have thought of it already :)