Make default tqdm dict overridable

[kuynzereb]

Resolves #629.

This PR moves default tqdm_dict definition from Trainer to LightningModule, so it can be overridden by the user. Now you just need to overwrite get_tqdm_dict() on the module to define your own default tqdm items.

Also I have removed batch_idx and gpu from the default tqdm dict and changed all batch/s to it/s as was suggested by @CarloLucibello.

@tullie, @williamFalcon, @Borda

[williamFalcon]

@kuynzereb fails on GPU tests...

self = LightningDataParallel(
  (module): LightningTestModel(
    (c_d1): Linear(in_features=784, out_features=1000, bias=Tru...)
    (c_d1_drop): Dropout(p=0.2, inplace=False)
    (c_d2): Linear(in_features=1000, out_features=10, bias=True)
  )
)
name = 'get_tqdm_dict'

    def __getattr__(self, name):
        if '_parameters' in self.__dict__:
            _parameters = self.__dict__['_parameters']
            if name in _parameters:
                return _parameters[name]
        if '_buffers' in self.__dict__:
            _buffers = self.__dict__['_buffers']
            if name in _buffers:
                return _buffers[name]
        if '_modules' in self.__dict__:
            modules = self.__dict__['_modules']
            if name in modules:
                return modules[name]
        raise AttributeError("'{}' object has no attribute '{}'".format(
>           type(self).__name__, name))
E       AttributeError: 'LightningDataParallel' object has no attribute 'get_tqdm_dict'



-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================= 11 failed, 102 passed, 23 warnings in 161.23s (0:02:41) ===

[kuynzereb]

I think I have fixed it. But actually I don't understand how did you get these fails, because travis shows no errors and I was not able to reproduce it locally...

[williamFalcon]

these are GPU tests. Travis doesn't run GPU tests

[kuynzereb]

Ah, okay, got it. Moreover several tests require more than 1 gpu, so it turned out that I can't run the full test-suite because now I have only 1 gpu on my machine :(

[Borda]

tackling #486

[CarloLucibello]

I would go with

if self.tqdm_metrics == None:
    return ref_model.get_tqdm_dict()
else:
    return self.tqdm_metrics

so that that the default dict can be completely overloaded

[kuynzereb]

But it is already can be completely overloaded :)
self.tqdm_metrics is not the default, it is totally determined by progress_bar dict from the training_end() and validation_end(). And model.get_tqdm_dict() adds some additional information like version number, gpu, etc. With this PR it is also totally controlled by the user.

So if you return an empty dict in model.get_tqdm_dict() and have no progress_bar dicts you will have a tqdm progress bar without any additional info.

[CarloLucibello]

right, sorry for the noise, I thought that get_tqdm_dict was not exposed. Is it discoverable? Should something be added somewhere in the docs?

[kuynzereb]

Well, now it can be found as one of the methods of LightningModule. Maybe we can add some information about how progress bar is composed, but I don't understand where it can be inserted in the current docs.

[williamFalcon]

@kuynzereb @CarloLucibello what's the verdict? @Borda

[Borda]

LGTM

[Borda]

make it as property like a tqdm_params?

[kuynzereb]

Yeah, at first I made it property. But then I accidentally found out that it will cause misleading error messages, if there is an attribute error in the user code. What I mean, if you make

@property
def tqdm_dict(self):
    _ = self.trainer.some_non_existing_attribute
    return {}

Then after

return dict(**ref_model.tqdm_dict, **self.tqdm_metrics)

you will get

AttributeError: 'CoolSystem' object has no attribute 'tqdm_dict'

I don't know how to fix it, but it seems to be the known issue: https://stackoverflow.com/questions/15348331/why-does-property-decorator-show-object-has-no-attribute

Overall, I decided to stay without property, so the user gets the clear error message if there is some error in his code.

[CarloLucibello]

looks fine