-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace a MisconfigurationException with warning in ModelCheckpoint callback #4560
Replace a MisconfigurationException with warning in ModelCheckpoint callback #4560
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4560 +/- ##
======================================
- Coverage 93% 93% -0%
======================================
Files 116 116
Lines 8855 8854 -1
======================================
- Hits 8248 8244 -4
- Misses 607 610 +3 |
with pytest.raises( | ||
MisconfigurationException, match=r'ModelCheckpoint\(save_last=True, monitor=None\) is not a valid.*' | ||
): | ||
ModelCheckpoint(dirpath=tmpdir, save_last=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can convert this to pytest.warns(UserWarning, ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great catch !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…allback (#4560) * replace MisconfigurationException with warning * update test * check raising UserWarning Co-authored-by: chaton <thomas@grid.ai> Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
What does this PR do?
I have a custom checkpoint callback that inherits from the
ModelCheckpoint
callback to supportevery_n_steps
by saving all(save_top_k=-1
andmonitor=None
) and skip if not every n steps ((global_step + 1) % every_n_steps != 0
)At the same time, I also want to store the last checkpoint(
save_last=True
) so that we could resume training from crash.However, this exception prevent me from doing so.
This PR replace the exception with a warning as discussed in #4335
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In in short, see following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃