Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest --collect-only fails when 'fail under' is set on a configuration #426

Closed
maurofaccenda opened this issue Aug 21, 2020 · 1 comment · Fixed by #511
Closed

pytest --collect-only fails when 'fail under' is set on a configuration #426

maurofaccenda opened this issue Aug 21, 2020 · 1 comment · Fixed by #511

Comments

@maurofaccenda
Copy link

maurofaccenda commented Aug 21, 2020

Summary

My pyproject.toml have the following sections (among other):

[tool.pytest.ini_options]
minversion = '6.0'
addopts = '--quiet --cov'

[tool.coverage.run]
source = ['my_module']

[tool.coverage.report]
fail_under = 75

And some tools (such as VSCode Python's plugin) depends on pytest --collect-only. However, when running that it fails as it doesn't actually run the tests and it displays a coverage way below the real value.

Expected vs actual result

Expected:
when running pytest --collect-only I'd expect it only to fail if for some reason it collect the list of tests, ignoring the fail under parameter.

Actual result:
pytest --collect-only fails stating the coverage didn't reach the goal.

Reproducer

git clone https://github.com/maurofaccenda/pytest-cov-issue.git
cd pytest-cov-issue/
python3 -m venv venv
source ./venv/bin/activate
pip install pip --upgrade
pip install -r requirements.txt
pytest  # succeeds
pytest --collect-only  # fails
echo $?  # prints 1

Versions

pip list output:

Package        Version
-------------- -------
attrs          20.1.0
coverage       5.2.1
iniconfig      1.0.1
more-itertools 8.4.0
packaging      20.4
pip            20.2.2
pkg-resources  0.0.0
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.1
pytest-cov     2.10.1
setuptools     44.0.0
six            1.15.0
toml           0.10.1
$ python --version
Python 3.8.2
$ pytest --version
pytest 6.0.1

Config

[tool.pytest.ini_options]
minversion = '6.0'
addopts = '--quiet --cov'

[tool.coverage.run]
source = ['my_module']

[tool.coverage.report]
fail_under = 75

Code

https://github.com/maurofaccenda/pytest-cov-issue

@maurofaccenda
Copy link
Author

Maybe this is related to pytest alone, but as it wouldn't happen without pytest-cov, I thought it would make sense to create the issue here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant