Fix sync_all_reduce to consider update->compute->update case

[sadra-barikbin]

Problem Description:
We get wrong result in the situation below, given that we are in a distributed config and SomeMetric has decorated its compute method with sync_all_reduce:

metric = SomeMetric() 
metric.update(y_pred, y)
correct_result = metric.compute()
metric.update(y_pred, y)
wrong_result = metric.compute()

This is the case because sync_all_reduce changes the state attributes to the accumulated values in all ranks, making them ready to call compute and calling update again doesn't make sense.

Solution (By @vfdev-5 ):
In sync_all_reduce, store the state attributes in temporary variables. Then do the collective ops on the state attributes. After that, call the decorated function. Finally, restore the state attributes and return the result of the decorated function.

• New tests are added (if a new feature is added)
• New doc strings: description and/or example code are in RST format
• Documentation is updated (if required)

[vfdev-5]

@sadra-barikbin let's write a test that fails on master and is fixed with this PR. In the test we have to check Metric attributes of type: tensor and scalar

[vfdev-5]

@sadra-barikbin the code is definitely broken, no point to update the branch every time

[vfdev-5]

@sadra-barikbin let's also remove self._is_reduced as now we do not need it anymore.

[sadra-barikbin]

Shall I remove reinit__is_reduced as well?

[sadra-barikbin]

Or you keep it for BC and just set _result to none if it exists?

[vfdev-5]

Or you keep it for BC and just set _result to none if it exists?

Yes, we should keep it for BC and also for reseting cached value. Later we can rename it with deprecation cycles etc.

[vfdev-5]

@sadra-barikbin please resolve the conflict for this PR

[vfdev-5]

LGTM, thanks @sadra-barikbin !