Skip to content

Commit

Permalink
Clear fs cache on session finish
Browse files Browse the repository at this point in the history
- avoids problems with cached modules during session shutdown
- see #866
  • Loading branch information
mrbean-bremen committed Aug 18, 2023
1 parent 841face commit cc67e1e
Show file tree
Hide file tree
Showing 6 changed files with 59 additions and 12 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/testsuite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ jobs:
run: |
pip install -r requirements.txt
pip install -U pytest==${{ matrix.pytest-version }}
pip install opentimelineio
pip install -e .
if [[ '${{ matrix.pytest-version }}' == '4.0.2' ]]; then
pip install -U attrs==19.1.0
fi
Expand Down
4 changes: 4 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ The released versions correspond to PyPI releases.

## Unreleased

### Fixes
* Clear the patched module cache on session shutdown (pytest only)
(see [#866](../../issues/866)). Added a class method `Patcher.cler_fs_cache`
for clearing the patched module cache.

## [Version 5.2.3](https://pypi.python.org/pypi/pyfakefs/5.2.3) (2023-07-10)
Adds compatibility with PyPy 3.10 and Python 3.12.
Expand Down
22 changes: 17 additions & 5 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -625,8 +625,7 @@ use_cache
.........
If True (the default), patched and non-patched modules are cached between tests
to avoid the performance hit of the file system function lookup (the
patching itself is reverted after each test). As this is a new
feature, this argument allows to turn it off in case it causes any problems:
patching itself is reverted after each test). This argument allows to turn it off in case it causes any problems:

.. code:: python
Expand All @@ -635,9 +634,22 @@ feature, this argument allows to turn it off in case it causes any problems:
fake_fs.create_file("foo", contents="test")
...
Please write an issue if you encounter any problem that can be fixed by using
this parameter. Note that this argument may be removed in a later version, if
no problems come up.
If using ``pytest``, the cache is always cleared before the final test shutdown, as there has been a problem
happening on shutdown related to removing the cached modules.
This does not happen for other test methods so far.

If you think you have encountered a similar problem with ``unittest``, you may try to clear the cache
during module shutdown using the class method for clearing the cache:

.. code:: python
from pyfakefs.fake_filesystem_unittest import Patcher
def tearDownModule():
Patcher.clear_fs_cache()
Please write an issue if you encounter any problem that can be fixed by using this parameter.

If you want to clear the cache just for a specific test instead, you can call
``clear_cache`` on the ``Patcher`` or the ``fake_filesystem`` instance:
Expand Down
18 changes: 12 additions & 6 deletions pyfakefs/fake_filesystem_unittest.py
Original file line number Diff line number Diff line change
Expand Up @@ -629,13 +629,19 @@ def __init__(
self._dyn_patcher: Optional[DynamicPatcher] = None
self._patching = False

def clear_cache(self) -> None:
@classmethod
def clear_fs_cache(cls) -> None:
"""Clear the module cache."""
self.__class__.CACHED_MODULES = set()
self.__class__.FS_MODULES = {}
self.__class__.FS_FUNCTIONS = {}
self.__class__.FS_DEFARGS = []
self.__class__.SKIPPED_FS_MODULES = {}
print("Clearing the cache")
cls.CACHED_MODULES = set()
cls.FS_MODULES = {}
cls.FS_FUNCTIONS = {}
cls.FS_DEFARGS = []
cls.SKIPPED_FS_MODULES = {}

def clear_cache(self) -> None:
"""Clear the module cache (convenience instance method)."""
self.__class__.clear_fs_cache()

def _init_fake_module_classes(self) -> None:
# IMPORTANT TESTING NOTE: Whenever you add a new module below, test
Expand Down
9 changes: 8 additions & 1 deletion pyfakefs/pytest_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ def my_fakefs_test(fs):
fs.create_file('/var/data/xx1.txt')
assert os.path.exists('/var/data/xx1.txt')
"""

import py
import pytest
from _pytest import capture
Expand All @@ -17,7 +18,7 @@ def my_fakefs_test(fs):
try:
from _pytest import pathlib
except ImportError:
pathlib = None
pathlib = None # type:ignore[assignment]

Patcher.SKIPMODULES.add(py)
Patcher.SKIPMODULES.add(pytest)
Expand Down Expand Up @@ -73,3 +74,9 @@ def fs_session(request):
patcher.setUp()
yield patcher.fs
patcher.tearDown()


@pytest.hookimpl(tryfirst=True)
def pytest_sessionfinish(session, exitstatus):
"""Make sure that the cache is cleared before the final test shutdown."""
Patcher.clear_fs_cache()
16 changes: 16 additions & 0 deletions pyfakefs/pytest_tests/segfault_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""
This is a regression test for #866 that shall ensure that
shutting down the test session after this specific call does not result
in a segmentation fault.
"""
import opentimelineio as otio


def test_empty_fs(fs):
pass


def test_create_clip(fs):
"""If the fs cache is not cleared during session shutdown, a segmentation fault
will happen during garbage collection of the cached modules."""
otio.core.SerializableObjectWithMetadata(metadata={})

0 comments on commit cc67e1e

Please sign in to comment.