Fix RMSLE metric

[Sordie]

What does this PR do?

Fixes #3162

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #3188 into master will increase coverage by 0%.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master   #3188   +/-   ##
======================================
  Coverage      90%     90%           
======================================
  Files          84      84           
  Lines        8008    8008           
======================================
+ Hits         7186    7187    +1     
+ Misses        822     821    -1     

[SkafteNicki]

We need some randomized tests against sklearn for the regression metrics. Something similar to:
https://github.com/PyTorchLightning/pytorch-lightning/blob/011235505544f95dacf42cffd545bb53dd4b6b15/tests/metrics/functional/test_classification.py#L38-L59
It would prevent such bugs.

[SkafteNicki]

This should test the remaining 4 metrics that are not already being tested against another framework:

from functools import partial
from math import sqrt

from sklearn.metrics import (
    mean_absolute_error as mae_sk,
    mean_squared_error as mse_sk,
    mean_squared_log_error as msle_sk
)

@pytest.mark.parametrize(['sklearn_metric', 'torch_metric'], [
    pytest.param(mae_sk, mae, id='mean_absolute_error'),
    pytest.param(mse_sk, mse, id='mean_squared_error'),
    pytest.param(partial(mse_sk, squared=False), rmse, id='root_mean_squared_error'),
    pytest.param(lambda x,y: sqrt(msle_sk(x,y)), rmsle, id='root_mean_squared_log_error')
])
def test_against_sklearn(sklearn_metric, torch_metric):
    """Compare PL metrics to sklearn version."""
    device = 'cuda' if torch.cuda.is_available() else 'cpu'

    # iterate over different label counts in predictions and target
    pred = torch.rand(300, device=device)
    target = torch.rand(300, device=device)

    sk_score = sklearn_metric(target.cpu().detach().numpy(),
                              pred.cpu().detach().numpy())
    sk_score = torch.tensor(sk_score, dtype=torch.float, device=device)
    pl_score = torch_metric(pred, target)
    assert torch.allclose(sk_score, pl_score)

[Borda]

This should test the remaining 4 metrics that are not already being tested against another framework:

cool, mind add it to this PR directly 👍

[pep8speaks]

Hello @Sordie! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-08-26 09:35:07 UTC