pytest incompatible with pytest-tldr

[heynemann]

Describe the bug

When running pytest with syrupy I get an exception.

To reproduce

My pip freeze reports the following pytest related packages (I can include the entire pip freeze if that's important):

pytest==6.2.5
pytest-asyncio==0.17.2
pytest-cov==3.0.0
pytest-forked==1.4.0
pytest-tldr==0.2.4
pytest-xdist==2.5.0
syrupy==1.7.0

I created a new test file called test_file.py:

def test_foo(snapshot):
    actual = "Some computed value!"
    assert actual == snapshot

When runnint with pytest tests/test_file.py I get the following output:

F
==============================================================================
FAIL: tests/test_file.py::test_foo
------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heynemann/dev/thumbor/tests/test_file.py", line 3, in test_foo
    assert actual == snapshot
AssertionError: assert [+ received] == [- snapshot]
  Snapshot 'test_foo' does not exist!
  + 'Some computed va...

  ...Full output truncated (1 line hidden), use '-vv' to show

------------------------------------------------------------------------------
Ran 1 tests in 0.01s

------------------- snapshot report summary --------------------
Traceback (most recent call last):
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/_pytest/main.py", line 304, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pytest_tldr.py", line 334, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/heynemann/.pyenv/versions/3.10.1/envs/thumbor/lib/python3.10/site-packages/syrupy/__init__.py", line 169, in pytest_terminal_summary
    for line in terminalreporter.config._syrupy.report.lines:
AttributeError: 'NoneType' object has no attribute 'lines'

Expected behavior

Expected the snapshot to be created.

Environment (please complete the following information):

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:        20.04
Codename:       focal

• Syrupy Version: 1.7.0
• Python Version: 3.10.1 (also tried with 3.9.9)
• Pytest version: 6.2.5

[noahnu]

This is intended behaviour. You don't have a snapshot and so it's correctly reporting a test failure.

If you expect snapshots to be generated, run pytest with the --snapshot-update cli flag.

[noahnu]

The attribute error at the bottom of your stack trace looks like a bug though. That should be a quick fix, but doesn't block you.

[heynemann]

That's the bug. It does block as that means I get exit code 1 in the test run, instead of 0 even if the test passes with a snapshot.

[noahnu]

I'm not sure I'm following. The expected behaviour is an exit code of 1. Your test did fail, you were missing a snapshot.

From the Motivation section in the README:

> Soundness: Snapshot tests should uncover even the most minute issues. Unlike other snapshot libraries, Syrupy will fail a test suite if a snapshot does not exist, not just on snapshot differences.

[noahnu]

It looks like the stack trace is only printed when using pytest-tldr though. Investigating

[noahnu]

Ah I see what you're saying. You get an exit code of 1 even when the snapshot is passing because of the tldr incompatibility.

[noahnu]

#583 should fix it. Essentially pytest-tldr was manually invoking a pytest hook which seems like bad practice. This was breaking an assumption around the hook order.

[tophat-opensource-bot]

🎉 This issue has been resolved in version 1.7.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[heynemann]

Thanks!!!