Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use tb_lineno to point to correct line in traceback #17

Merged
merged 4 commits into from
Sep 27, 2023
Merged

Use tb_lineno to point to correct line in traceback #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3fe64dc
Use tb_lineno to point to correct line in traceback
alexmojaki Sep 17, 2023
e07b153
Keep the original co_firstlineno
alexmojaki Sep 17, 2023
089cf8a
Adjust co_firstlineno so that the pytest includes the correct context.
alexmojaki Sep 20, 2023
e07ec3d
Loosen testing result.outlines
alexmojaki Sep 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions pytest_examples/traceback.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from __future__ import annotations as _annotations

import sys
import traceback
from types import CodeType, FrameType, TracebackType
from typing import TYPE_CHECKING

Expand All @@ -21,27 +20,28 @@ def create_example_traceback(exc: Exception, module_path: str, example: CodeExam
# f_code.co_posonlyargcount was added in 3.8
return None
frames = []
for frame, _ in traceback.walk_tb(exc.__traceback__):
tb = exc.__traceback__
while tb is not None:
frame = tb.tb_frame
if frame.f_code.co_filename == module_path:
frames.append(create_custom_frame(frame, example))
frames.append((create_custom_frame(frame, example), tb.tb_lasti, tb.tb_lineno + example.start_line))
samuelcolvin marked this conversation as resolved.
Show resolved Hide resolved
tb = tb.tb_next

frames.reverse()
new_tb = None
for altered_frame in frames:
new_tb = TracebackType(
tb_next=new_tb, tb_frame=altered_frame, tb_lasti=altered_frame.f_lasti, tb_lineno=altered_frame.f_lineno
)
for altered_frame, lasti, lineno in frames:
new_tb = TracebackType(tb_next=new_tb, tb_frame=altered_frame, tb_lasti=lasti, tb_lineno=lineno)
return new_tb


def create_custom_frame(frame: FrameType, example: CodeExample) -> FrameType:
"""
Create a new frame that mostly matches `frame` but with a filename from `example` and line number
altered to match the example.
Create a new frame that mostly matches `frame` but with a code object that has
a filename from `example` and adjusted an adjusted first line number
so that pytest shows the correct code context in the traceback.

Taken mostly from https://naleraphael.github.io/blog/posts/devlog_create_a_builtin_frame_object/
With the CodeType creation inspired by https://stackoverflow.com/a/16123158/949890. However, we use
`frame.f_lineno` for the line number instead of `f_code.co_firstlineno` as that seems to work.
With the CodeType creation inspired by https://stackoverflow.com/a/16123158/949890.
"""
import ctypes

Expand Down Expand Up @@ -77,7 +77,7 @@ def create_custom_frame(frame: FrameType, example: CodeExample) -> FrameType:
str(example.path),
f_code.co_name,
f_code.co_qualname,
frame.f_lineno + example.start_line,
f_code.co_firstlineno + example.start_line,
f_code.co_lnotab,
f_code.co_exceptiontable,
)
Expand All @@ -95,7 +95,7 @@ def create_custom_frame(frame: FrameType, example: CodeExample) -> FrameType:
f_code.co_varnames,
str(example.path),
f_code.co_name,
frame.f_lineno + example.start_line,
f_code.co_firstlineno + example.start_line,
f_code.co_lnotab,
)

Expand Down
15 changes: 10 additions & 5 deletions tests/test_run_examples.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,11 @@ def test_find_run_examples(example: CodeExample, eval_example: EvalExample):
result = pytester.runpytest('-p', 'no:pretty', '-v')
result.assert_outcomes(passed=1, failed=1)

# assert 'my_file_9_13.py:12: AssertionError' in '\n'.join(result.outlines)
assert result.outlines[-8:-3] == [
assert result.outlines[-11].startswith('_ _ _ _ ')
assert result.outlines[-10:-3] == [
'',
' a = 1',
' b = 2',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is caused by changing to f_code.co_firstlineno, which seems to help pytest include more context from the original markdown file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand this properly now. Previously, setting co_firstlineno to frame.f_lineno + example.start_line was saying that the code started at the problematic line (ignoring the nuance about f_lineno vs tb_lineno) so pytest would show the failing line (so things appeared to work) but it thought that was the start of the code so it didn't show any context before that.

Leaving co_firstlineno unchanged meant it included more context, but too much.

f_code.co_firstlineno + example.start_line works correctly as you'd expect. That's the only way that this expanded test passes, with pytest showing the exact correct context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#19 doesn't pass this test because the code in line_adjusted_source still starts at the first line, even if it's just padding comments, so the full example file gets included as context for the top level frame.

'> assert a + b == 4',
'E AssertionError',
'',
Expand Down Expand Up @@ -224,7 +226,10 @@ def test_run_directly(tmp_path, eval_example):
x = 4

def div(y):
return x / y
try:
return x / y
finally:
str(y)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the line where the traceback would point without the change, as it's the last line the frame executed. tb_lineno points to where the exception happened, two lines up.


div(2)
div(0)"""
Expand All @@ -244,10 +249,10 @@ def div(y):

# debug(exc_info.traceback)
assert exc_info.traceback[-1].frame.code.path == md_file
assert exc_info.traceback[-1].lineno == 6
assert exc_info.traceback[-1].lineno == 7

assert exc_info.traceback[-2].frame.code.path == md_file
assert exc_info.traceback[-2].lineno == 9
assert exc_info.traceback[-2].lineno == 12


def test_print_sub(pytester: pytest.Pytester):
Expand Down