fallback to native traceback when handling ExceptionGroup (take 2) [SQUASH]

[jakkdl]

Picking up work on and therefore closes #10071 to temporarily address #9159

Improvements over the previous PR:

• Handles BaseExceptionGroup
• Handles exception chains properly, using excinfo_ instead of excinfo, and sets reprcrash
• fixed tox.ini
• rewrote tests extending them, previous ones also had regex bugs

I'm not entirely sure how to handle the envlist in tox.ini, since hypothesis currently depends on exceptiongroup, making an extra environment for it redundant. I also wanted a test for py311 without exceptiongroup installed, but it got messy setting up a separate testenv without extras=testing. An alternative is a 3.11-only test that modifies _pytest._code.code.ExceptionGroups removing exceptiongroup.BaseExceptionGroup from it.

The test is still somewhat WIP, I want some extra matching lines to differentiate the parametrized variables, but thought I'd push my current work for initial review.

@Zac-HD

[jakkdl]

failing code coverage is partly for the 3.11-only test, and partly for the code written for when exceptiongroup is not installed.

[jakkdl]

also shoutout to @Cheukting for giving a good base to work off of. I'll add you to AUTHORS if you want :)

[Cheukting]

Thank you @jakkdl 😊

[Zac-HD]

Two comments below; otherwise both tests and implementation look good to me. I've also confirmed that it works locally 👍

Definitely add Cheuk to the authors list though, it's important to give OSS credit (and copyright acknowledgement!) where it's due.

[Zac-HD]

Tagging @RonnyPfannschmidt for review too because I'm pretty close to the whole ExceptionGroup project, between PEP-678 and implementing support in Hypothesis.

[jakkdl]

Fixed comments, although Cheukting is already in AUTHORS 😅 Also changed the tests a little bit.
Adding exceptiongroup to the pre-commit-config is just for line 64, from exceptiongroup import BaseExceptionGroup, so maybe simpler to just #type: ignore that line instead.

I managed to figure out how to add a tox testenv to test py37 without exceptiongroup installed, but it's quite a hassle and doesn't seem worth it just to test ~2 lines of code.

[Zac-HD]

Looks great to me - thanks @jakkdl!

If I don't hear any objections (@nicoddemus @RonnyPfannschmidt), I'll merge this on Friday.

[nicoddemus]

Left a few comments, please take a look.

[bluetech]

Since exceptiongroup hasn't had a stable release yet and has had some breaking changes in the recent releases, let's ask exceptiongroup's developer:

@agronholm - pytest is considering adding a dependency on exceptiongroup for Python versions before 3.11. pytest is a popular project so the dependency will add a lot of installs of exceptiongroup. Is this OK with you? If so, do you think it's OK to depend on version exceptiongroup>=1.0.0rc8 or should we wait for a final release? Thanks.

[agronholm]

The only reason I've held back the final release is to ensure that it matches the behavior of Python 3.11 in its final release.
That said, I would like to resolve the last two open issues (agronholm/exceptiongroup#18 and agronholm/exceptiongroup#14) before making a final release.

And yes, I'm okay with pytest depending on exceptiongroup. But the latter of the open issues would need to be resolved for pytest which, IIRC, installs its own exception hook, yes?

[bluetech]

But the latter of the open issues would need to be resolved for pytest which, IIRC, installs its own exception hook, yes?

pytest installs threading.excepthook and sys.unraisablehook, but not sys.excepthook. Is that OK?

[agronholm]

If it overrides those hooks, then it needs some way to manually render exception groups – something the exceptiongroup package doesn't provide yet. I'm not sure how to even provide such functionality in a way that doesn't involve patching TracebackException. It's something that needs to be researched. I would welcome any help with this.

[Zac-HD]

If it overrides those hooks, then it needs some way to manually render exception groups – something the exceptiongroup package doesn't provide yet. I'm not sure how to even provide such functionality in a way that doesn't involve patching TracebackException.

I'd have thought that calling the traceback.format_exception() in (without loss of generality) pytest's hook function would work, since that ultimately delegates to the patched-by-exceptiongroup TracebackException methods?

And that's what Pytest's sys.unraisablehook and threading.excepthook currently do, without going through the fancier formatting that this PR works around, so I think it already works.

[agronholm]

I'd have thought that calling the traceback.format_exception() in (without loss of generality) pytest's hook function would work, since that ultimately delegates to the patched-by-exceptiongroup TracebackException methods?

You're probably right – I am no so knowledgeable about the inner workings of the traceback module. I just adapted trio's MultiError patching to exceptiongroup. To be honest, I don't even know why pytest patches those two specific hooks but NOT sys.excepthook.

[Zac-HD]

I don't even know why pytest patches those two specific hooks but NOT sys.excepthook.

(I'm reconstructing rather than recalling, but) we don't need to register a sys.excepthook because we'll always catch the exception coming out of the user's test function. We do want to register a simple hook for the others; they just convert from the default "print to stderr" to "emit a warning containing the same text" - I guess someone had a reason to prefer that at some point, maybe warnings capture is more reliable around threads? 🤷‍♂️

[Zac-HD]

As a high-level comment, I'd also observe that this PR seems like a strict improvement on the status quo, so I propose that we merge it (without the 3.11-exceptiongroup CI task) and move any remaining discussion to a new issue 🙂

[jakkdl]

Yeah not much meaningful has changed after reviews, though the code is cleaner now ✨
Removed the added toxenv from CI, and added comments to tox and pre-commit configs to avoid future confusion and make continued work with exceptiongroups easier.

[nicoddemus]

Looks great, thanks everyone for chiming in.

Let's please use squash when merging this. 👍 (I've added [SQUASH] to the title as a reminder).

[Zac-HD]

🎉🎉

Thanks to everyone involved! I'm looking forward to using this 😁