Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for iterable datasets when val_check_interval=1.0 #1283

Merged

Conversation

ethanwharris
Copy link
Member

@ethanwharris ethanwharris commented Mar 29, 2020

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • not yet, see below
  • Did you write any new necessary tests?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #1252

  • had to make a method with will look ahead one batch - which may be a memory issue (but since we usually use multi-threaded data loading anyway should be fine) - can't see any other way to do it
  • docs will need to be updated after Update docs iterable datasets #1281

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@ethanwharris ethanwharris requested a review from a team March 29, 2020 17:22
@codecov
Copy link

codecov bot commented Mar 29, 2020

Codecov Report

Merging #1283 into master will increase coverage by <1%.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master   #1283    +/-   ##
=======================================
+ Coverage      92%     92%   +<1%     
=======================================
  Files          61      61            
  Lines        3153    3164    +11     
=======================================
+ Hits         2886    2897    +11     
  Misses        267     267

@williamFalcon williamFalcon merged commit ab09faa into Lightning-AI:master Mar 29, 2020
@ethanwharris ethanwharris deleted the feature/iterable_val_check branch March 29, 2020 19:34
@Borda
Copy link
Member

Borda commented Mar 29, 2020

the multiline bool expression is quite hard to read...

alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Mar 30, 2020
…ing-AI#1283)

* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
@Borda Borda added the feature Is an improvement or enhancement label Mar 30, 2020
@Borda Borda added this to the 0.7.2 milestone Mar 30, 2020
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 1, 2020
…ing-AI#1283)

* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 3, 2020
…ing-AI#1283)

* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 3, 2020
…ing-AI#1283)

* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 3, 2020
…ing-AI#1283)

* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
@Borda Borda modified the milestones: v0.7., v0.7.x Apr 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validation every epoch with non-finite dataloader
3 participants