Improve error handling in runner.py job.run(...)
#8761
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
4 times, most recently
from
bfc53cf
to
6587920
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #8761 +/- ##
==========================================
+ Coverage 91.44% 91.53% +0.09%
==========================================
Files 344 345 +1
Lines 21122 21252 +130
==========================================
+ Hits 19314 19454 +140
+ Misses 1808 1798 -10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
from
07228b8
to
8a9720c
Compare
jonathan-eq
added
maintenance
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
2 times, most recently
from
58f3fc2
to
f2f17d7
Compare
xjules
reviewed
Sep 25, 2024
|
|
||
| # exit_code is 0 | ||
|
|
||
| if self.job_data.get("error_file") and os.path.exists( | ||
| self.job_data["error_file"] | ||
| ): | ||
| yield exited_message.with_error( | ||
| exited_message = Exited(self, exit_code) | ||
| return exited_message.with_error( |
There was a problem hiding this comment.
return?? previously we yielded ....
There was a problem hiding this comment.
Yes, but this is a helper function for creating the message that is being yielded from the calling method.
|
Lots of errors. Wondering if it is related that you have changed some |
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
from
f2f17d7
to
baa4c51
Compare
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
from
baa4c51
to
b688ca8
Compare
It should be fine. The methods returning values are just for creating the messages that are being yielded in the |
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
5 times, most recently
from
e401754
to
a641fdf
Compare
This commit wraps the `job.run(...)` method in a try-except block making sure the `Exited` message is sent if the job_runner crashes. This fixes the bug where ert is hanging when unhandled exceptions are raised in the job runner.
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
from
a641fdf
to
72e838e
Compare
jonathan-eq
added
improvement
JHolba
approved these changes
Oct 8, 2024
jonathan-eq
force-pushed
the
improve-forward-model-runner-exception-handling
branch
from
72e838e
to
71daee6
Compare
jonathan-eq
added
release-notes:improvement
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Resolves #8738
Approach
This commit wraps the
job.run(...)method in a try-except block making sure theExitedmessage is sent if the job_runner crashes. This fixes the bug where ert is hanging when unhandled exceptions are raised in the job runner.The PR also contains a commit that refactors the mega-method
job.run(...)into separate methods, making it more readable.(Screenshot of new behavior in GUI if applicable)
git rebase -i main --exec 'pytest tests/unit_tests -n logical -m "not integration_test"')When applicable