Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report reason for pytest skips #3942

Merged
merged 2 commits into from
Jul 5, 2019

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Jul 3, 2019

Changes proposed in this pull request:

  • Show a summary of skipped tests, helps identify gaps in testing
  • Add as config so no need to modify lines in several CI scripts

For example:

=========================== short test summary info ===========================
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: Numpy is not installed
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: images are not collected
SKIPPED [7] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: olefile package not installed
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: netpbm not available
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: requires macOS
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: djpeg not available
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: cjpeg not available
SKIPPED [1] Tests\test_file_jpeg2k.py:175: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:181: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:152: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:169: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:163: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:53: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:48: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:93: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:43: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:121: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:111: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:63: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:75: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:79: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:70: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:193: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:97: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:101: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:105: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:139: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:31: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:87: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:83: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_jpeg2k.py:187: JPEG 2000 support not available
SKIPPED [1] Tests\test_file_palm.py:40: Palm P image is wrong
SKIPPED [8] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: requires Unix or macOS
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: Test requires opening tempfile twice
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: Failing on AppVeyor when run from subprocess, not from shell
SKIPPED [1] Tests\test_image_fromqimage.py:26: Qt bindings are not installed
SKIPPED [1] Tests\test_image_fromqimage.py:38: Qt bindings are not installed
SKIPPED [1] Tests\test_image_fromqimage.py:42: Qt bindings are not installed
SKIPPED [1] Tests\test_image_fromqimage.py:30: Qt bindings are not installed
SKIPPED [1] Tests\test_image_fromqimage.py:34: Qt bindings are not installed
SKIPPED [1] Tests\test_image_quantize.py:18: libimagequant support not available
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: current implementation isn't precise enough
SKIPPED [1] Tests\test_imagedraw.py:633: failing
SKIPPED [2] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: requires Unix or macOS
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: requires Python 3.x on Windows
SKIPPED [38] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: Raqm not Available
SKIPPED [13] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: Raqm Library is not installed.
SKIPPED [1] Tests\test_imageqt.py:91: Qt bindings are not installed
SKIPPED [1] Tests\test_imageqt.py:87: Qt bindings are not installed
SKIPPED [1] Tests\test_imageqt.py:59: Qt bindings are not installed
SKIPPED [1] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: Win32 does not call map_buffer
SKIPPED [12] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: Numpy is not installed
SKIPPED [1] C:\vp\pypy2\site-packages\_pytest\unittest.py:94: Pyroma is not installed
SKIPPED [1] Tests\test_qt_image_fromqpixmap.py:13: Qt bindings are not installed
SKIPPED [1] Tests\test_qt_image_toqimage.py:35: Qt bindings are not installed
SKIPPED [1] Tests\test_qt_image_toqimage.py:70: Qt bindings are not installed
SKIPPED [1] Tests\test_qt_image_toqpixmap.py:11: Qt bindings are not installed
SKIPPED [1] ..\pypy2.7-v7.1.1-win32\lib-python\2.7\unittest\case.py:60: os.path support for Paths added in 3.6
================= 1218 passed, 137 skipped in 119.34 seconds ==================

The option:

$ pytest --help
...
  -r chars              show extra test summary info as specified by chars:
                        (f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed,
                        (p)assed, (P)assed with output, (a)ll except passed
                        (p/P), or (A)ll. (w)arnings are enabled by default
                        (see --disable-warnings).

Edit: updated to the more usual -rs; -r s was a workaround for a Python 3.8 pytest bug, since fixed in pytest 5.0.1: pytest-dev/pytest#5523.

@hugovk hugovk added the Testing label Jul 3, 2019
@hugovk hugovk mentioned this pull request Jul 4, 2019
@radarhere radarhere merged commit 6568829 into python-pillow:master Jul 5, 2019
@hugovk hugovk deleted the report-pytest-skips branch July 6, 2019 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants