Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5/n: Extract reference model call to plugins/accelerators #4773

Merged
merged 30 commits into from
Nov 23, 2020

Conversation

SeanNaren
Copy link
Contributor

@SeanNaren SeanNaren commented Nov 19, 2020

What does this PR do?

Closes #4753. Final API change to allow custom DDP wrapper logic when extracting reference model.

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In in short, see following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?

Make sure you had fun coding 🙃

@pep8speaks
Copy link

pep8speaks commented Nov 19, 2020

Hello @SeanNaren! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-11-23 16:38:09 UTC

@justusschock
Copy link
Member

@SeanNaren I like it. This seems like a trivial function now, but could you maybe add a test for it to ensure that it still works, if we may come to more complicated cases?

@SeanNaren
Copy link
Contributor Author

@SeanNaren I like it. This seems like a trivial function now, but could you maybe add a test for it to ensure that it still works, if we may come to more complicated cases?

Absolutely, will do. Also ddp plugin tests are failing, need to ensure this logic is overridden correctly in the tests.

@codecov
Copy link

codecov bot commented Nov 19, 2020

Codecov Report

Merging #4773 (d9289fb) into master (6831ba9) will increase coverage by 0%.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master   #4773   +/-   ##
======================================
  Coverage      93%     93%           
======================================
  Files         118     118           
  Lines        9018    9020    +2     
======================================
+ Hits         8389    8393    +4     
+ Misses        629     627    -2     

Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a DDP test ?

tests/trainer/properties/test_get_model.py Show resolved Hide resolved
@Borda Borda requested a review from tchaton November 20, 2020 23:38
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just concern about the methods which really do not require self if they shall be rather a class method or static

pytorch_lightning/plugins/ddp_plugin.py Outdated Show resolved Hide resolved
pytorch_lightning/accelerators/dp_accelerator.py Outdated Show resolved Hide resolved
pytorch_lightning/plugins/ddp_plugin.py Outdated Show resolved Hide resolved
pytorch_lightning/accelerators/dp_accelerator.py Outdated Show resolved Hide resolved
@SeanNaren SeanNaren requested a review from Borda November 22, 2020 11:40
pytorch_lightning/plugins/ddp_plugin.py Outdated Show resolved Hide resolved
@SeanNaren SeanNaren changed the title Sharded Plugin 5/n: Extract reference model call to plugins/accelerators 5/n: Extract reference model call to plugins/accelerators Nov 23, 2020
@SeanNaren SeanNaren merged commit 404af43 into master Nov 23, 2020
@SeanNaren SeanNaren deleted the feature/817-fairscale-5n branch November 23, 2020 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed Generic distributed-related topic refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants