Linux Bad File Descriptor when watched directory is removed

[ElaPl]

Hello

I tested watchdog on Linux (Ubuntu) and Windows (Win10).

Versions

Linux::

Python 3.6.9
watchdog-1.0.2

Windows

Python 3.7.9
watchdog-1.0.2

Test scenario is as below:

1. Create directory X
2. Watch directory X and handle events using FileSystemEventHandler
3. Remove directory X
4. Unschedule observer (using observer.unschedule_all or observer.unschedule)
5. Stop observer and join

Issue:

Linux:

1. When observer.unschedule_all() is called, OSError(9, "Bad File Descriptor") is raised.
2. When observer.unschedule(path) is called, KeyError is raised

Windows: Everything works fine.

Example source code

import os
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

# Linux path
WATCHED_DIR = "/home/eplaszcz/PycharmProjects/project_dir"

# Windows path
# WATCHED_DIR = "C:\\Users\\eplaszcz\\Documents\\watchdog_test\\project_dir"


def run_watchdog():
    observer = Observer()
    watched_dir = WATCHED_DIR
    event_handler = FileSystemEventHandler()

    if not os.path.isdir(watched_dir):
        os.mkdir(WATCHED_DIR)

    observer.start()

    print("Schedule target watcher for %s" % watched_dir)
    observer.schedule(event_handler, path=watched_dir, recursive=True)

    while os.path.isdir(watched_dir):
        time.sleep(2)
        os.rmdir(WATCHED_DIR)

    print("Active watchers: {}".format(observer._watches))

    print("Directory removed, unschedule")
    observer.unschedule_all()

    print("Observer unscheduled")
    print("Active watchers: {}".format(observer._watches))

    print("Directory removed, stop")
    observer.stop()

    print("Directory removed, join")
    observer.join()


if __name__ == '__main__':
    run_watchdog()

Linux output when using observer.unschedule_all

python3 main.py 
Schedule target watcher for C:\Users\eplaszcz\Documents\watchdog_test\project_dir
Active watchers: {<ObservedWatch: path=C:\Users\eplaszcz\Documents\watchdog_test\project_dir, is_recursive=True>}
Directory removed, unschedule
Traceback (most recent call last):
  File "main.py", line 46, in <module>
    run_watchdog()
  File "main.py", line 33, in run_watchdog
    observer.unschedule_all()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/api.py", line 357, in unschedule_all
    self._clear_emitters()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/api.py", line 231, in _clear_emitters
    emitter.stop()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 81, in stop
    self.on_thread_stop()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/inotify.py", line 121, in on_thread_stop
    self._inotify.close()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 50, in close
    self.stop()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 81, in stop
    self.on_thread_stop()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 46, in on_thread_stop
    self._inotify.close()
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 277, in close
    os.close(self._inotify_fd)
OSError: [Errno 9] Bad file descriptor

Linux output when using observer.unschedule(watched_dir)

python3 main.py 
Schedule target watcher for /home/eplaszcz/PycharmProjects/project_dir
Active watchers: {<ObservedWatch: path=/home/eplaszcz/PycharmProjects/project_dir, is_recursive=True>}
Directory removed, unschedule
Traceback (most recent call last):
  File "main.py", line 46, in <module>
    run_watchdog()
  File "main.py", line 33, in run_watchdog
    observer.unschedule(WATCHED_DIR)
  File "/home/eplaszcz/PycharmProjects/watchdogFileDescriptor/venv/lib/python3.6/site-packages/watchdog/observers/api.py", line 347, in unschedule
    emitter = self._emitter_for_watch[watch]
KeyError: '/home/eplaszcz/PycharmProjects/project_dir'

** Windows output **

py main.py
Schedule target watcher for C:\Users\eplaszcz\Documents\watchdog_test\project_dir
Active watchers: {<ObservedWatch: path=C:\Users\eplaszcz\Documents\watchdog_test\project_dir, is_recursive=True>}
Directory removed, unschedule
Observer unscheduled
Active watchers: set()
Directory removed, stop
Directory removed, join

[samschott]

Is it possible that #727 already fixed this? I recall making some modifications to the inotify and winapi backends to have uniform behaviour when the watched path is deleted.

[samschott]

@ElaPl, do you still see this issue in the master branch or only with the latest release?

[samschott]

Actually, this issue is not fixed by #727. We handled this in the tests by catching the OSError.