Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validation metrics not logging properly #3761

Closed
jeremyjordan opened this issue Oct 1, 2020 · 1 comment
Closed

validation metrics not logging properly #3761

jeremyjordan opened this issue Oct 1, 2020 · 1 comment
Labels
bug Something isn't working help wanted Open to be worked on

Comments

@jeremyjordan
Copy link
Contributor

🐛 Bug

Validation metrics aren't being logged to the Trainer's logger.

To Reproduce

See Colab notebook: https://colab.research.google.com/drive/1O93pAD_cfSuVH6izyzrsn_xEe8v_2O7n?usp=sharing

Based on this test: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/tests/trainer/test_eval_loop_logging_1_0.py

Code sample

from pytorch_lightning import Trainer
from tests.base.deterministic_model import DeterministicModel
import os
import torch
from pytorch_lightning.loggers import CSVLogger


class TestModel(DeterministicModel):
    def training_step(self, batch, batch_idx):
        acc = self.step(batch, batch_idx)
        acc = acc + batch_idx
        self.log('a', acc)
        self.log('b', acc, on_step=True, on_epoch=True)
        self.training_step_called = True
        return acc
    def validation_step(self, batch, batch_idx):
        acc = self.step(batch, batch_idx)
        acc = acc + batch_idx
        self.log('c', acc, on_step=False, on_epoch=True)
        self.log('d', acc, on_step=False, on_epoch=True)
        self.validation_step_called = True
        return acc
    def validation_step_end(self, acc):
        pass
    def validation_epoch_end(self, outputs):
        self.log('g', torch.tensor(2, device=self.device), on_epoch=True)
        self.validation_epoch_end_called = True
    def backward(self, trainer, loss, optimizer, optimizer_idx):
        loss.backward()


model = TestModel()
logger = CSVLogger("logs", name="my_exp_name")
trainer = Trainer(
    logger=logger,
    limit_train_batches=2,
    limit_val_batches=2,
    max_epochs=2,
    row_log_interval=1,
    weights_summary=None,
)
trainer.fit(model)

Expected behavior

Validation metric keys aren't present.

@jeremyjordan jeremyjordan added bug Something isn't working help wanted Open to be worked on labels Oct 1, 2020
@jeremyjordan
Copy link
Contributor Author

Fixed by #3763

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on
Projects
None yet
Development

No branches or pull requests

1 participant