[WIP] uneven input support for DDP

[otaj]

What does this PR do?

WIP for Uneven input support for DDP

Fixes #3325

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

[carmocca]

This should be opt-in (Maybe you set enable=True by default whilst it's WIP)

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need further help see our docs: https://pytorch-lightning.readthedocs.io/en/latest/generated/CONTRIBUTING.html#pull-request or ask the assistance of a core contributor here or on Slack. Thank you for your contributions.

[anth2o]

Hello !
Is this PR still ongoing ?

[otaj]

Hello ! Is this PR still ongoing ?

Hi @anth2o! Currently, there is no progress. It is somewhat still on my radar, but I've been busy with higher priority items lately.

We still want to finalize it, but it doesn't have the highest priority, honestly, so there is no ETA.

[anth2o]

Thanks for your answer, I'll try to find a solution on my end to handle this

[ssharpe42]

Is there anything on this topic being worked on currently? What is the current best suggestion in the case of using iterable datasets on separate (possibly unequal) sharded data for each gpu in training and evaluation w/ torchmetrics?

[ssharpe42]

Forgot to tag @carmocca @otaj. Also @anth2o did you find a solution here?

[ivandariojr]

This would be really useful to get consistent evaluations of datasets that may not be divisible by the number of devices. Has there been any progress on this?