diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 98aba5381fc..e8f56db2b53 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -533,6 +533,7 @@ pep-0674.rst @vstinner pep-0675.rst @jellezijlstra pep-0676.rst @Mariatta pep-0677.rst @gvanrossum +pep-0678.rst @iritkatriel # ... # pep-0754.txt # ... diff --git a/pep-0678.rst b/pep-0678.rst new file mode 100644 index 00000000000..3d96f5f395d --- /dev/null +++ b/pep-0678.rst @@ -0,0 +1,270 @@ +PEP: 678 +Title: Enriching Exceptions with Notes +Author: Zac Hatfield-Dodds +Sponsor: Irit Katriel +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Requires: 654 +Created: 20-Dec-2021 +Python-Version: 3.11 +Post-History: + + +Abstract +======== +Exception objects are typically initialized with a message that describes the +error which has occurred. Because further information may be available when the +exception is caught and re-raised, this PEP proposes to add a ``.__note__`` +attribute and update the builtin traceback formatting code to include it in +the formatted traceback following the exception string. + +This is particularly useful in relation to :pep:`654` ``ExceptionGroup`` s, which +make previous workarounds ineffective or confusing. Use cases have been identified +in the standard library, Hypothesis package, and common code patterns with retries. + + +Motivation +========== +When an exception is created in order to be raised, it is usually initialized +with information that describes the error that has occurred. There are cases +where it is useful to add information after the exception was caught. +For example, + +- testing libraries may wish to show the values involved in a failing assertion, + or the steps to reproduce a failure (e.g. ``pytest`` and ``hypothesis`` ; example below). +- code with retries may wish to note which iteration or timestamp raised which + error - especially if re-raising them in an ``ExceptionGroup`` +- programming environments for novices can provide more detailed descriptions + of various errors, and tips for resolving them (e.g. ``friendly-traceback`` ). + +Existing approaches must pass this additional information around while keeping +it in sync with the state of raised, and potentially caught or chained, exceptions. +This is already error-prone, and made more difficult by :pep:`654` ``ExceptionGroup`` s, +so the time is right for a built-in solution. We therefore propose to add a mutable +field ``__note__`` to ``BaseException`` , which can be assigned a string - and +if assigned, is automatically displayed in formatted tracebacks. + + +Example usage +------------- + + >>> try: + ... raise TypeError('bad type') + ... except Exception as e: + ... e.__note__ = 'Add some information' + ... raise + ... + Traceback (most recent call last): + File "", line 2, in + TypeError: bad type + Add some information + >>> + +When collecting exceptions into an exception group, we may want +to add context information for the individual errors. In the following +example with `Hypothesis' proposed support for ExceptionGroup +`__, each +exception includes a note of the minimal failing example:: + + from hypothesis import given, strategies as st, target + + @given(st.integers()) + def test(x): + assert x < 0 + assert x > 0 + + + + Exception Group Traceback (most recent call last): + | File "test.py", line 4, in test + | def test(x): + | + | File "hypothesis/core.py", line 1202, in wrapped_test + | raise the_error_hypothesis_found + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + | ExceptionGroup: Hypothesis found 2 distinct failures. + +-+---------------- 1 ---------------- + | Traceback (most recent call last): + | File "test.py", line 6, in test + | assert x > 0 + | ^^^^^^^^^^^^ + | AssertionError: assert -1 > 0 + | + | Falsifying example: test( + | x=-1, + | ) + +---------------- 2 ---------------- + | Traceback (most recent call last): + | File "test.py", line 5, in test + | assert x < 0 + | ^^^^^^^^^^^^ + | AssertionError: assert 0 < 0 + | + | Falsifying example: test( + | x=0, + | ) + +------------------------------------ + + +Specification +============= + +``BaseException`` gains a new mutable attribute ``__note__`` , which defaults to +``None`` and may have a string assigned. When an exception with a note is displayed, +the note is displayed immediately after the exception. + +Assigning a new string value overrides an existing note; if concatenation is desired +users are responsible for implementing it with e.g.:: + + e.__note__ = msg if e.__note__ is None else e.__note__ + "\n" + msg + +It is an error to assign a non-string-or-``None`` value to ``__note__`` , +or to attempt to delete the attribute. + +``BaseExceptionGroup.subgroup`` and ``BaseExceptionGroup.split`` +copy the ``__note__`` of the original exception group to the parts. + + +Backwards Compatibility +======================= + +System-defined or "dunder" names (following the pattern ``__*__`` ) are part of the +language specification, with unassigned names reserved for future use and subject +to breakage without warning [1]_. + +We are also unaware of any code which *would* be broken by adding ``__note__`` ; +assigning to a ``.__note__`` attribute already *works* on current versions of +Python - the note just won't be displayed with the traceback and exception message. + + + +How to Teach This +================= + +The ``__note__`` attribute will be documented as part of the language standard, +and explained as part of the tutorial "Errors and Exceptions" [2]_. + + + +Reference Implementation +======================== + +``BaseException.__note__`` was implemented in [3]_ and released in CPython 3.11.0a3, +following discussions related to :pep:`654`. [4]_ [5]_ [6]_ + + + +Rejected Ideas +============== + +Use ``print()`` (or ``logging`` , etc.) +--------------------------------------- +Reporting explanatory or contextual information about an error by printing or logging +has historically been an acceptable workaround. However, we dislike the way this +separates the content from the exception object it refers to - which can lead to +"orphan" reports if the error was caught and handled later, or merely significant +difficulties working out which explanation corresponds to which error. +The new ``ExceptionGroup`` type intensifies these existing challenges. + +Keeping the ``__note__`` attached to the exception object, like the traceback, +eliminates these problems. + + +``raise Wrapper(explanation) from err`` +--------------------------------------- +An alternative pattern is to use exception chaining: by raising a 'wrapper' exception +containing the context or explanation ``from`` the current exception, we avoid the +separation challenges from ``print()`` . However, this has two key problems. + +First, it changes the type of the exception, which is often a breaking change for +downstream code. We consider *always* raising a ``Wrapper`` exception unacceptably +inelegant; but because custom exception types might have any number of required +arguments we can't always create an instance of the *same* type with our explanation. +In cases where the exact exception type is known this can work, such as the standard +library ``http.client`` code [7]_, but not for libraries which call user code. + +Second, exception chaining reports several lines of additional detail, which are +distracting for experienced users and can be very confusing for beginners. +For example, six of the eleven lines reported for this simple example relate to +exception chaining, and are unnecessary with ``BaseException.__note__`` : + +.. code-block:: python + + class Explanation(Exception): + def __str__(self): + return "\n" + str(self) + + try: + raise AssertionError("Failed!") + except Exception as e: + raise Explanation("You can reproduce this error by ...") from e + +.. code-block:: + + $ python example.py + Traceback (most recent call last): + File "example.py", line 6, in + raise AssertionError(why) + AssertionError: Failed! + # These lines are + The above exception was the direct cause of the following exception: # confusing for new + # users, and they + Traceback (most recent call last): # only exist due + File "example.py", line 8, in # to implementation + raise Explanation(msg) from e # constraints :-( + Explanation: # Hence this PEP! + You can reproduce this error by ... + + +Subclass Exception and add ``__note__`` downstream +-------------------------------------------------- +Traceback printing is built into the C code, and reimplemented in pure Python in +traceback.py. To get ``err.__note__`` printed from a downstream implementation +would *also* require writing custom traceback-printing code; while this could +be shared between projects and reuse some pieces of traceback.py we prefer to +implement this once, upstream. + +Custom exception types could implement their ``__str__`` method to include our +proposed ``__note__`` semantics, but this would be rarely and inconsistently +applicable. + + +Store notes in ``ExceptionGroup`` s +----------------------------------- +Initial discussions proposed making a more focussed change by thinking about how to +associate messages with the nested exceptions in ``ExceptionGroup`` s, such as a list +of notes or mapping of exceptions to notes. However, this would force a remarkably +awkward API and retains a lesser form of the cross-referencing problem discussed +under "use ``print()`` " above; if this PEP is rejected we prefer the status quo. +Finally, of course, ``__note__`` is not only useful with ``ExceptionGroup`` s! + + + +References +========== + +.. [1] https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers +.. [2] https://github.com/python/cpython/pull/30158 +.. [3] https://github.com/python/cpython/pull/29880 +.. [4] https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/9 +.. [5] https://github.com/python/cpython/pull/28569#discussion_r721768348 +.. [6] https://bugs.python.org/issue45607 +.. [7] https://github.com/python/cpython/blob/69ef1b59983065ddb0b712dac3b04107c5059735/Lib/http/client.py#L596-L597 + + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: