Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a convenience function for floating-point comparisons #1441

Merged
merged 16 commits into from
Mar 15, 2016
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Jason R. Coombs
Joshua Bronson
Jurko Gospodnetić
Katarzyna Jachim
Kale Kundert
Kevin Cox
Lee Kamentsky
Lukas Bednar
Expand Down
3 changes: 2 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
namespace in which your doctests run.
Thanks `@milliams`_ for the complete PR (`#1428`_).

*
* New ``approx()`` function for easily comparing floating-point numbers in
tests.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a "Thanks @kalekundert for the complete PR". 😄


*

Expand Down
113 changes: 111 additions & 2 deletions _pytest/python.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
""" Python test discovery, setup and run of test functions. """

import fnmatch
import functools
import inspect
Expand Down Expand Up @@ -261,7 +262,8 @@ def pytest_namespace():
return {
'fixture': fixture,
'yield_fixture': yield_fixture,
'raises' : raises,
'raises': raises,
'approx': approx,
'collect': {
'Module': Module, 'Class': Class, 'Instance': Instance,
'Function': Function, 'Generator': Generator,
Expand Down Expand Up @@ -1203,7 +1205,8 @@ def getlocation(function, curdir):
# builtin pytest.raises helper

def raises(expected_exception, *args, **kwargs):
""" assert that a code block/function call raises ``expected_exception``
"""
Assert that a code block/function call raises ``expected_exception``
and raise a failure exception otherwise.

This helper produces a ``ExceptionInfo()`` object (see below).
Expand Down Expand Up @@ -1336,6 +1339,112 @@ def __exit__(self, *tp):
self.excinfo.__init__(tp)
return issubclass(self.excinfo.type, self.expected_exception)

# builtin pytest.approx helper

class approx(object):
"""
Assert that two numbers (or two sets of numbers) are equal to each
other within some margin.

Due to the intricacies of floating-point arithmetic, numbers that we would
intuitively expect to be the same are not always so::

>>> 0.1 + 0.2 == 0.3
False

This problem is commonly encountered when writing tests, e.g. when making
sure that floating-point values are what you expect them to be. One way to
deal with this problem is to assert that two floating-point numbers are
equal to within some appropriate margin::

>>> abs((0.1 + 0.2) - 0.3) < 1e-6
True

However, comparisons like this are tedious to write and difficult to
understand. Furthermore, absolute comparisons like the one above are
usually discouraged in favor of relative comparisons, which can't even be
easily written on one line. The ``approx`` class provides a way to make
floating-point comparisons that solves both these problems::

>>> from pytest import approx
>>> 0.1 + 0.2 == approx(0.3)
True

``approx`` also makes is easy to compare ordered sets of numbers, which
would otherwise be very tedious::

>>> (0.1 + 0.2, 0.2 + 0.4) == approx((0.3, 0.6))
True

By default, ``approx`` considers two numbers to be equal if the relative
error between them is less than one part in a million (e.g. ``1e-6``).
Relative error is defined as ``abs(x - a) / x`` where ``x`` is the value
you're expecting and ``a`` is the value you're comparing to. This
definition breaks down when the numbers being compared get very close to
zero, so ``approx`` will also consider two numbers to be equal if the
absolute difference between them is less than one part in a trillion (e.g.
``1e-12``).

Both the relative and absolute error thresholds can be changed by passing
arguments to the ``approx`` constructor::

>>> 1.0001 == approx(1)
False
>>> 1.0001 == approx(1, rel=1e-3)
True
>>> 1.0001 == approx(1, abs=1e-3)
True

Note that if you specify ``abs`` but not ``rel``, the comparison will not
consider the relative error between the two values at all. In other words,
two numbers that are within the default relative error threshold of 1e-6
will still be considered unequal if they exceed the specified absolute
error threshold. If you specify both ``abs`` and ``rel``, the numbers will
be considered equal if either threshold is met::

>>> 1 + 1e-8 == approx(1)
True
>>> 1 + 1e-8 == approx(1, abs=1e-12)
False
>>> 1 + 1e-8 == approx(1, rel=1e-6, abs=1e-12)
True
"""

def __init__(self, expected, rel=None, abs=None):
self.expected = expected
self.max_relative_error = rel
self.max_absolute_error = abs

def __repr__(self):
from collections import Iterable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not do import collections at the top of the file instead?

utf_8 = lambda s: s.encode('utf-8') if sys.version_info.major == 2 else s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this is not supported in py26: sys.version_info.major. You will have to use sys.version_info[0] == 2.

plus_minus = lambda x: utf_8(u'{0} \u00b1 {1:.1e}'.format(x, self._get_margin(x)))

if isinstance(self.expected, Iterable):
return ', '.join([plus_minus(x) for x in self.expected])
else:
return plus_minus(self.expected)

def __eq__(self, actual):
from collections import Iterable
expected = self.expected
almost_eq = lambda a, x: abs(x - a) < self._get_margin(x)

if isinstance(actual, Iterable) and isinstance(expected, Iterable):
return all(almost_eq(a, x) for a, x in zip(actual, expected))
else:
return almost_eq(actual, expected)

def _get_margin(self, x):
margin = self.max_absolute_error or 1e-12

if self.max_relative_error is None:
if self.max_absolute_error is not None:
return margin

return max(margin, x * (self.max_relative_error or 1e-6))


#
# the basic pytest Function item
#
Expand Down
7 changes: 6 additions & 1 deletion doc/en/builtin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ Examples at :ref:`assertraises`.

.. autofunction:: deprecated_call

Comparing floating point numbers
--------------------------------

.. autoclass:: approx

Raising a specific test outcome
--------------------------------------

Expand All @@ -48,7 +53,7 @@ you can rather use declarative marks, see :ref:`skipping`.
.. autofunction:: _pytest.skipping.xfail
.. autofunction:: _pytest.runner.exit

fixtures and requests
Fixtures and requests
-----------------------------------------------------

To mark a fixture function:
Expand Down
33 changes: 33 additions & 0 deletions testing/python/approx.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# encoding: utf-8

import pytest
import doctest

class MyDocTestRunner(doctest.DocTestRunner):

def __init__(self):
doctest.DocTestRunner.__init__(self)

def report_failure(self, out, test, example, got):
raise AssertionError("'{}' evaluates to '{}', not '{}'".format(
example.source.strip(), got.strip(), example.want.strip()))


class TestApprox:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use @pytest.mark.parametrize here to separate tests from each other (so e.g. other tests still run if one of them fails), but I'm okay with the current form as well.


def test_approx_doctests(self):
parser = doctest.DocTestParser()
test = parser.get_doctest(
pytest.approx.__doc__,
{'approx': pytest.approx},
pytest.approx.__name__,
None, None,
)
runner = MyDocTestRunner()
runner.run(test)

def test_repr_string(self):
# Just make sure the Unicode handling doesn't raise any exceptions.
print(pytest.approx(1.0))
print(pytest.approx([1.0, 2.0, 3.0]))