Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-24412: Adds cleanUps for setUpClass and setUpModule. #9190

Merged
merged 7 commits into from
Nov 9, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions Doc/library/unittest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1448,6 +1448,39 @@ Test cases

.. versionadded:: 3.1

.. classmethod:: addClassCleanup(function, *args, **kwargs)

Add a function to be called after :meth:`tearDownClass` to cleanup
resources used during the test class. Functions will be called in reverse
order to the order they are added (:abbr:`LIFO (last-in, first-out)`).
They are called with any arguments and keyword arguments passed into
:meth:`addClassCleanup` when they are added.

If :meth:`setUpClass` fails, meaning that :meth:`tearDownClass` is not
called, then any cleanup functions added will still be called.

.. versionadded:: 3.8


.. classmethod:: doClassCleanups()

This method is called unconditionally after :meth:`tearDownClass`, or
after :meth:`setUpClass` if :meth:`setUpClass` raises an exception.

It is responsible for calling all the cleanup functions added by
:meth:`addCleanupClass`. If you need cleanup functions to be called
*prior* to :meth:`tearDownClass` then you can call
:meth:`doCleanupsClass` yourself.

:meth:`doCleanupsClass` pops methods off the stack of cleanup
functions one at a time, so it can be called at any time.

.. versionadded:: 3.8






.. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)

Expand Down Expand Up @@ -2268,6 +2301,38 @@ module will be run and the ``tearDownModule`` will not be run. If the exception
:exc:`SkipTest` exception then the module will be reported as having been skipped
instead of as an error.

To add cleanup code that must be run even in the case of an exception, use
``addModuleCleanup``:


.. function:: addModuleCleanup(function, *args, **kwargs)

Add a function to be called after :func:`tearDownModule` to cleanup
resources used during the test class. Functions will be called in reverse
order to the order they are added (:abbr:`LIFO (last-in, first-out)`).
They are called with any arguments and keyword arguments passed into
:meth:`addModuleCleanup` when they are added.

If :meth:`setUpModule` fails, meaning that :func:`tearDownModule` is not
called, then any cleanup functions added will still be called.

.. versionadded:: 3.8


.. function:: doModuleCleanups()

This function is called unconditionally after :func:`tearDownModule`, or
after :func:`setUpModule` if :func:`setUpModule` raises an exception.

It is responsible for calling all the cleanup functions added by
:func:`addCleanupModule`. If you need cleanup functions to be called
*prior* to :func:`tearDownModule` then you can call
:func:`doModuleCleanups` yourself.

:func:`doModuleCleanups` pops methods off the stack of cleanup
functions one at a time, so it can be called at any time.

.. versionadded:: 3.8

Signal Handling
---------------
Expand Down
9 changes: 9 additions & 0 deletions Doc/whatsnew/3.8.rst
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,15 @@ unicodedata
is in a specific normal form. (Contributed by Max Belanger and David Euresti in
:issue:`32285`).

unittest
--------

* Added :func:`~unittest.addModuleCleanup()` and
:meth:`~unittest.TestCase.addClassCleanup()` to unittest to support
cleanups for :func:`~unittest.setUpModule()` and
:meth:`~unittest.TestCase.setUpClass()`.
(Contributed by Lisa Roach in :issue:`24412`.)

venv
----

Expand Down
7 changes: 4 additions & 3 deletions Lib/unittest/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,17 @@ def testMultiply(self):
'TextTestRunner', 'TestLoader', 'FunctionTestCase', 'main',
'defaultTestLoader', 'SkipTest', 'skip', 'skipIf', 'skipUnless',
'expectedFailure', 'TextTestResult', 'installHandler',
'registerResult', 'removeResult', 'removeHandler']
'registerResult', 'removeResult', 'removeHandler',
'addModuleCleanup']

# Expose obsolete functions for backwards compatibility
__all__.extend(['getTestCaseNames', 'makeSuite', 'findTestCases'])

__unittest = True

from .result import TestResult
from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf,
skipUnless, expectedFailure)
from .case import (addModuleCleanup, TestCase, FunctionTestCase, SkipTest, skip,
skipIf, skipUnless, expectedFailure)
from .suite import BaseTestSuite, TestSuite
from .loader import (TestLoader, defaultTestLoader, makeSuite, getTestCaseNames,
findTestCases)
Expand Down
46 changes: 45 additions & 1 deletion Lib/unittest/case.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,30 @@ def testPartExecutor(self, test_case, isTest=False):
def _id(obj):
return obj


_module_cleanups = []
warsaw marked this conversation as resolved.
Show resolved Hide resolved
def addModuleCleanup(function, *args, **kwargs):
"""Same as addCleanup, except the cleanup items are called even if
setUpModule fails (unlike tearDownModule)."""
_module_cleanups.append((function, args, kwargs))


def doModuleCleanups():
"""Execute all module cleanup functions. Normally called for you after
tearDownModule."""
exceptions = []
while _module_cleanups:
function, args, kwargs = _module_cleanups.pop()
try:
function(*args, **kwargs)
except Exception as exc:
exceptions.append(exc)
if exceptions:
# Swallows all but first exception. If a multi-exception handler
# gets written we should use that here instead.
raise exceptions[0]
Copy link
Member

@warsaw warsaw Sep 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally get why you have to do it this way, although it would be nice if we had trio's MultiError in the stdlib. :) It's unfortunate that we have to throw away all the other exceptions, but I don't know where you can stash them. E.g. IIUC, instance cleanups stash the exceptions on self.errors, but there's no such place on modules (unless this code adds it, which, yuck :)). The other alternative is to create something like a simple MultiError, add the exceptions to that, and then raise the MultiError (maybe only if len(exceptions) > 1?).

It seems like a lot of extra complication for hopefully an, um, exceptional case, so maybe punt on that for now. IOW, I'm cool with this, unless you have any other ideas.



def skip(reason):
"""
Unconditionally skip a test.
Expand Down Expand Up @@ -390,6 +414,8 @@ class TestCase(object):

_classSetupFailed = False

_class_cleanups = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit (maybe!): In this case, the blank line between these two class attributes may not be needed.


def __init__(self, methodName='runTest'):
"""Create an instance of the class that will use the named test
method when executed. Raises a ValueError if the instance does
Expand Down Expand Up @@ -445,6 +471,12 @@ def addCleanup(self, function, *args, **kwargs):
Cleanup items are called even if setUp fails (unlike tearDown)."""
self._cleanups.append((function, args, kwargs))

@classmethod
def addClassCleanup(cls, function, *args, **kwargs):
"""Same as addCleanup, except the cleanup items are called even if
setUpClass fails (unlike tearDownClass)."""
cls._class_cleanups.append((function, args, kwargs))

def setUp(self):
"Hook method for setting up the test fixture before exercising it."
pass
Expand Down Expand Up @@ -651,9 +683,21 @@ def doCleanups(self):
function(*args, **kwargs)

# return this for backwards compatibility
# even though we no longer us it internally
# even though we no longer use it internally
return outcome.success

@classmethod
def doClassCleanups(cls):
"""Execute all class cleanup functions. Normally called for you after
tearDownClass."""
cls.tearDown_exceptions = []
while cls._class_cleanups:
function, args, kwargs = cls._class_cleanups.pop()
try:
function(*args, **kwargs)
except Exception as exc:
cls.tearDown_exceptions.append(sys.exc_info())

def __call__(self, *args, **kwds):
return self.run(*args, **kwds)

Expand Down
60 changes: 50 additions & 10 deletions Lib/unittest/suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,18 @@ def _handleClassSetUp(self, test, result):
raise
currentClass._classSetupFailed = True
className = util.strclass(currentClass)
errorName = 'setUpClass (%s)' % className
self._addClassOrModuleLevelException(result, e, errorName)
self._createClassOrModuleLevelException(result, e,
'setUpClass',
className)
finally:
_call_if_exists(result, '_restoreStdout')
if currentClass._classSetupFailed is True:
currentClass.doClassCleanups()
if len(currentClass.tearDown_exceptions) > 0:
for exc in currentClass.tearDown_exceptions:
self._createClassOrModuleLevelException(
result, exc[1], 'setUpClass', className,
info=exc)

def _get_previous_module(self, result):
previousModule = None
Expand Down Expand Up @@ -199,21 +207,37 @@ def _handleModuleFixture(self, test, result):
try:
setUpModule()
except Exception as e:
try:
case.doModuleCleanups()
except Exception as exc:
self._createClassOrModuleLevelException(result, exc,
'setUpModule',
currentModule)
if isinstance(result, _DebugResult):
raise
result._moduleSetUpFailed = True
errorName = 'setUpModule (%s)' % currentModule
self._addClassOrModuleLevelException(result, e, errorName)
self._createClassOrModuleLevelException(result, e,
'setUpModule',
currentModule)
finally:
_call_if_exists(result, '_restoreStdout')

def _addClassOrModuleLevelException(self, result, exception, errorName):
def _createClassOrModuleLevelException(self, result, exc, method_name,
parent, info=None):
errorName = f'{method_name} ({parent})'
self._addClassOrModuleLevelException(result, exc, errorName, info)

def _addClassOrModuleLevelException(self, result, exception, errorName,
info=None):
error = _ErrorHolder(errorName)
addSkip = getattr(result, 'addSkip', None)
if addSkip is not None and isinstance(exception, case.SkipTest):
addSkip(error, str(exception))
else:
result.addError(error, sys.exc_info())
if not info:
result.addError(error, sys.exc_info())
else:
result.addError(error, info)

def _handleModuleTearDown(self, result):
previousModule = self._get_previous_module(result)
Expand All @@ -235,10 +259,17 @@ def _handleModuleTearDown(self, result):
except Exception as e:
if isinstance(result, _DebugResult):
raise
errorName = 'tearDownModule (%s)' % previousModule
self._addClassOrModuleLevelException(result, e, errorName)
self._createClassOrModuleLevelException(result, e,
'tearDownModule',
previousModule)
finally:
_call_if_exists(result, '_restoreStdout')
try:
case.doModuleCleanups()
except Exception as e:
self._createClassOrModuleLevelException(result, e,
'tearDownModule',
previousModule)

def _tearDownPreviousClass(self, test, result):
previousClass = getattr(result, '_previousTestClass', None)
Expand All @@ -261,10 +292,19 @@ def _tearDownPreviousClass(self, test, result):
if isinstance(result, _DebugResult):
raise
className = util.strclass(previousClass)
errorName = 'tearDownClass (%s)' % className
self._addClassOrModuleLevelException(result, e, errorName)
self._createClassOrModuleLevelException(result, e,
'tearDownClass',
className)
finally:
_call_if_exists(result, '_restoreStdout')
previousClass.doClassCleanups()
if len(previousClass.tearDown_exceptions) > 0:
for exc in previousClass.tearDown_exceptions:
className = util.strclass(previousClass)
self._createClassOrModuleLevelException(result, exc[1],
'tearDownClass',
className,
info=exc)


class _ErrorHolder(object):
Expand Down
Loading