Progress bar missing with litdata.StreamingDataset and wrong number of steps in an epoch

[yhl48]

🐛 Bug

There are two separate issues here.

When training a model using litdata.StreamingDataset, the tqdm progress bar shows {steps}/? and the estimated time is missing.

Moreover, the total number of steps in an epoch seems to be independent of the number of GPUs. Instead of having total_steps = num_samples / (num_gpus * batch_size), the log returns total_steps = num_samples / batch_size

Expected behavior

The progress bar should show the estimated time and the fraction of steps that have been completed.

total_steps = num_samples / (num_gpus * batch_size)

cc @tchaton

[github-actions]

Hi! thanks for your contribution!, great first issue!

[yhl48]

I might be stating the obvious here but the issue seems to originate from this line where self.trainer.num_training_batches == inf

[yhl48]

related issue Lightning-AI/pytorch-lightning#15734

[yhl48]

With regard to the the data distribution across gpus, I believe this line

litdata/src/litdata/streaming/dataset.py

Line 167 in a09de86

self.distributed_env = _DistributedEnv.detect()

should be called again in __iter__ since the ddp process is initialised in Trainer, which is triggered after StreamingDataset is initialised, rendering this line

litdata/src/litdata/utilities/env.py

Line 48 in a09de86

if torch.distributed.is_available() and torch.distributed.is_initialized():

to always be False

@tchaton