Enable coverage reporting via codecov

[blueyed]

No description provided.

[blueyed]

Problem with py37:

___________________________________ test_ok ____________________________________
    def test_ok():
        app = TestApp(CGIApplication({}, script='ok.cgi', path=[data_dir]))
>       res = app.get('')
tests/test_cgiapp.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py37-coverage/lib/python3.7/site-packages/paste/wsgilib.py:351: in raw_interactive
    app_iter = application(basic_environ, start_response)
.tox/py37-coverage/lib/python3.7/site-packages/paste/lint.py:170: in lint_app
    iterator = application(environ, start_response_wrapper)
.tox/py37-coverage/lib/python3.7/site-packages/paste/cgiapp.py:104: in __call__
    stderr=environ['wsgi.errors'])
.tox/py37-coverage/lib/python3.7/site-packages/paste/cgiapp.py:256: in proc_communicate
    stderr.write(data)
.tox/py37-coverage/lib/python3.7/site-packages/paste/lint.py:221: in write
    self.errors.write(s)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <paste.wsgilib.ErrorRaiser object at 0x7f3fb70ccbe0>
value = b'Error in atexit._run_exitfuncs:\nTraceback (most recent call last):\n  File "/home/travis/build/cdent/paste/.tox/py3...9, in exists\n    os.stat(path)\nTypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType\n'
    def write(self, value):
        if not value:
            return
        raise AssertionError(
>           "No errors should be written (got: %r)" % value)
E       AssertionError: No errors should be written (got: b'Error in atexit._run_exitfuncs:\nTraceback (most recent call last):\n  File "/home/travis/build/cdent/paste/.tox/py37-coverage/lib/python3.7/genericpath.py", line 19, in exists\n    os.stat(path)\nTypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType\n')

https://travis-ci.org/cdent/paste/jobs/448095432#L537

[blueyed]

Not sure where this exists(None) call is coming from.

[blueyed]

Fixed in coveragepy 4.5.2a1 (not released, ref: nedbat/coveragepy@58b210a).

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@36e5b8b). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #10   +/-   ##
=========================================
  Coverage          ?   15.03%           
=========================================
  Files             ?      125           
  Lines             ?    14746           
  Branches          ?     2502           
=========================================
  Hits              ?     2217           
  Misses            ?    12492           
  Partials          ?       37

Flag	Coverage Δ
#py27	14.68% <ø> (?)
#py35	14.71% <ø> (?)
#py36	14.71% <ø> (?)
#pypy	14.58% <ø> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 36e5b8b...a72dce4. Read the comment docs.

[blueyed]

Should be good now, but should get squash-merged, please.

[blueyed]

This is not required anymore when using the "path" already.
The error was due to paste/ being used, but with tox it would use the package from .tox/… anyway.
See nedbat/coveragepy#268.
The proper fix here would be to use a "src" based layout, i.e. move "paste" to "src/paste", but not sure if it is worth it.

[blueyed]

Given the outcome/review I will adjust this here.

[blueyed]

So what about this?
I would say to go back to just "coverage" here.
Will push this already.

[cdent]

Yeah, go ahead and fix that in whatever way is right.

[cdent]

Could you add a human-oriented coverage tox env that allows someone to check coverage before pushing? Something like:

• rm .coverage and rm -r cover
• ran the tests with coverage turned on
• coverage html -d cover

Doesn't have to be in the pull request, could be a different one.

[blueyed]

Yes, could be useful, although tox -e py37-coverage should be enough and then allows you to use your preferred reporting, e.g. I would rather use coverage report … instead of the html report.

[blueyed]

..and pytest-cov does the reporting already to the terminal btw.

In general a Makefile might be better suited though, since otherwise with a separate env you get a duplicated one:

cover: PY=py37
cover:
	tox -e $(PY)-coverage
	.tox/$(PY)-coverage/bin/coverage html -d $@

Anyway, feedback via PRs is better in general though (since it covers all envs).

[cdent]

Anyway, feedback via PRs is better in general though (since it covers all envs).

The thing I'm trying to encourage is people checking their coverage before they ever push because when someone finally makes a pull request they should think it is ready. Which tooling we have to encourage that doesn't really matter to me, just so long as such tooling is available.

[blueyed]

So I guess running e.g. tox -e py37-coverage alone would be good then?
If you want to add it to docs, you can also mention to run coverage html -d cover afterwards for an HTML report I guess.

[cdent]

I reckon we can go ahead and merge this, and work any other details later.