doctest terminates when accessing __wrapped__ raises an error

[LasseSchuirmann]

BPO	25998
Nosy	@stevendaprano, @bitdancer, @jstasiak, @BoboTiG
PRs	gh-70186: Fix doctest badly handling unwrapable objects #14756
Files	shell: The sample shell session to reproduce unwrapable_ob.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-01-02.22:30:43.610>
labels = ['type-bug', 'tests']
title = 'doctest terminates when accessing __wrapped__ raises an error'
updated_at = <Date 2019-07-13.16:45:13.678>
user = 'https://bugs.python.org/LasseSchuirmann'

bugs.python.org fields:

activity = <Date 2019-07-13.16:45:13.678>
actor = 'Tiger-222'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2016-01-02.22:30:43.610>
creator = 'Lasse Schuirmann'
dependencies = []
files = ['41477', '48477']
hgrepos = []
issue_num = 25998
keywords = ['patch']
message_count = 8.0
messages = ['257378', '257379', '257387', '257467', '257478', '257479', '347791', '347845']
nosy_count = 5.0
nosy_names = ['steven.daprano', 'r.david.murray', 'jstasiak', 'Lasse Schuirmann', 'Tiger-222']
pr_nums = ['14756']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25998'
versions = ['Python 3.5', 'Python 3.6']

[LasseSchuirmann]

You can see this when importing the Flask request object in a file that is doctested. The request object will throw a RuntimeError when one tries to access any attribute. Doctest tries to inspect.unwrap all objects in the file in order to find out if they're doctestable functions - and apparently only catches the AttributeError exception. Thus it crashes :)

[LasseSchuirmann]

(Closed) issue at the flask repository of this: pallets/flask#1680

Also IIRC this worked back with Python 3.4 but do not consider this a fact yet. (At least the tests ran on Fedora...)

[stevendaprano]

doctest doesn't crash -- it is a regular exception, not a crash. "Crash" means "Hard crashes of the Python interpreter – possibly with a core dump or a Windows error box." In other words, a segmentation fault or other low-level crash, not an exception.

I'm not convinced that this is a problem with doctest, it looks to me like a broken-by-design flaw in requests. If you try to access an attribute that doesn't exist, you should get AttributeError, not RuntimeError. I don't understand the justification given on the requests tracker for why the attribute access fails, but I would expect that any exception other than AttributeError, or a subclass of such, is a clear bug in requests.

I don't think it is doctest's responsibility to have special handling for an ill-designed proxy class, but I'll hold off closing the ticket for a few days in case anyone makes a good argument for why this is a doctest bug.

[LasseSchuirmann]

Sorry for giving it the wrong category.

I personally think both projects should solve this. IMO it would be nice if doctest is robust against modules that for some reason (be they ill designed or not; I guess if you search a lot you find a project that needs to raise a custom exception when accessing __wrapped__). At the same time I do think the Flask request object could handle this better but lack the ability to explain that to them with substantial reasoning.

The problem is that both of you don't want to solve this problem at their side, @steven.daprano, would you mind speaking with the Flask people directly? I'm not affiliated to them.

[bitdancer]

No, I don't think doctest should be made robust against RuntimeErrors. doctest's purpose is to check for bugs, and IMO this is a bug in flask. In particular, it is not doctest that flask is not cooperating with, it is hasattr. It used to be that hasattr would mask errors raised, and this caused a number of both debugging and production problems, to the point where hasattr was not considered safe to use. But it eventually got fixed: http://www.gossamer-threads.com/lists/python/dev/856859

[bitdancer]

Gah. Having hit send I of course had second thoughts. doctest is suppose to *report* bugs, so there *is* a doctest bug here: it should be capturing that and probably other errors and reporting them, instead of just producing a traceback, I think.

[BoboTiG]

Working on it at EuroPython 2019.

FTR I tested on Python 3.6.9, 3.7.4 and 3.9.0a0 without issue.
I think this was fixed with rtweeks/werkzeug@c643980.

But the doctest issue remains and this is on what I will focus.

[BoboTiG]

Attached a simple reproduction script.

[davidism]

This is an issue with Werkzeug's proxy wrappers, see pallets/werkzeug#2485. When a proxy is unbound, any attribute access on it will raise an error. Doctest calls inspect.unwrap on everything in globals, and so it fails when there is a werkzeug.LocalProxy object.

Sphinx uses inspect.unwrap as well, but it guards it in a try/except Exception block so that access doesn't cause failure. It looks like doctest would be fine doing the same.

LocalProxy is always wrapping something, but it's not clear that it's always correct for it to set __wrapped__ when the thing being wrapped is not a function. That's the workaround I'm going to add for now, but I think it should still be addressed in doctest as well.