stop and join watchdog observer #11706
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is in accordance with the example. For inotify at least the `.stop()` call ends up calling the removed unschedule call anyways. https://github.com/gorakhargosh/watchdog/tree/72f2eb7203659c11c4f0703c818dcb88fe527e99#example-api-usage
|
Tracking flakes while rerunning this. https://github.com/Chia-Network/chia-blockchain/runs/6646220657?check_suite_focus=true |
|
PyPI went down for a bit. |
|
https://github.com/Chia-Network/chia-blockchain/runs/6646942280?check_suite_focus=true |
|
https://github.com/Chia-Network/chia-blockchain/runs/6647224077?check_suite_focus=true |
paninaro
approved these changes
May 31, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Draft for:
Note possible upstream fix for the issue at gorakhargosh/watchdog#895 being tested in #11710.
This is in accordance with the watchdog example. For inotify at least the
.stop()call ends up calling the remove unschedule call anyways.https://github.com/gorakhargosh/watchdog/tree/72f2eb7203659c11c4f0703c818dcb88fe527e99#example-api-usage
This appears to be only used in test code. This lowers the bar for trying this out in
mainto see how it holds up but also raises the question of why we aren't doing any shutdown at all for normal runs, seemingly.This was found while looking into the following traceback we occasionally get in CI.
____________ ERROR at teardown of TestWalletSync.test_almost_recent ____________ venv/lib/python3.10/site-packages/_pytest/runner.py:338: in from_call result: Optional[TResult] = func() venv/lib/python3.10/site-packages/_pytest/runner.py:259: in <lambda> lambda: ihook(item=item, **kwds), when=when, reraise=reraise venv/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) venv/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) venv/lib/python3.10/site-packages/_pytest/threadexception.py:88: in pytest_runtest_teardown yield from thread_exception_runtest_hook() venv/lib/python3.10/site-packages/_pytest/threadexception.py:73: in thread_exception_runtest_hook warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg)) E pytest.PytestUnhandledThreadExceptionWarning: Exception in thread Thread-44 E E Traceback (most recent call last): E File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/threading.py", line 1009, in _bootstrap_inner E self.run() E File "/home/runner/work/chia-blockchain/chia-blockchain/venv/lib/python3.10/site-packages/watchdog/observers/inotify_buffer.py", line 88, in run E inotify_events = self._inotify.read_events() E File "/home/runner/work/chia-blockchain/chia-blockchain/venv/lib/python3.10/site-packages/watchdog/observers/inotify_c.py", line 285, in read_events E event_buffer = os.read(self._inotify_fd, event_buffer_size) E OSError: [Errno 9] Bad file descriptorI don't have any historical data on how often this was happening so the several CI reruns below aren't any sort of proof that the issue is actually fixed. I just figured I'd give it a chance to still pop up and also collect a list of other flakes that are still hanging around. So far it looks like CI is in a pretty good state right now in terms of flakes. But yeah, the bad file descriptor issue did still show up with just this change.