multiple options to addopts

[kodonnell]

I might be missing something, but it seems the only way to pass multiple options to pytest are by using a single --addopts and wrapping the multiple optoins in quotes e.g.

python setup.py test --addopts '--cov=plugin.pyx --cov-config=.coveragerc'

Is this expected? If so, it might help to tweak the user_options definition to explain this. (As an aside, it wasn't clear to me how to even use one argument: --addopts arg=val or --addopts arg val etc.)

[jaraco]

Is this expected?

Yes.

It seems non-obvious to me how one could capture the nuances of this pass-through in a option description. I note that this format also matches that of the 'addopts' value in pytest.ini.

I tried adding a reference to this URL, but in --help, but I don't like the rendering:

  --addopts      Additional options to be passed verbatim to the pytest
                 runner, same as with
                 http://doc.pytest.org/en/latest/customize.html#adding-default
                 -options

One couldn't copy that URL to the clipboard and open it without editing it first. At the same time, I don't want to try to repeat that detail in the quick help.

Perhaps the solution is to put more details in the readme. Or maybe there's an elegant way to capture the details in the option description. Or maybe using a URL shortener would work there.

What do you recommend?

[kodonnell]

Actually, that link still left me a tab confused. I think it's because here we're specifying command line options inside of a command line option (as opposed to command line options inside of a config variable). E.g. here I would be worried about writing --addopts --arg=value, because normal behaviour is for this to be interpreted as two separate options. This ambiguity doesn't arise in the pytest.ini case, as the addopts aren't specified at the command line.

What do you recommend?

I'm not sure what's possible, as I'm pretty new to setuptools. That said, I'd be tempted to try:

• have something which catches all extra options, and fires them to pytest. Basically the same functionality as this, but without the --addopts or quoting. I had a quick look into how setuptools did it, but gave up ... so I'm not sure if it's possible.
• dynamically add pytest options to user_options, and fire them through that way. I'm not sure if you can dynamically strip options from pytest (and plugins).
• move to using a configuration file e.g. setup.cfg. (And avoid collisions as are happening between pytest-cov and pytest by having a nice separate section e.g. [pytest:pytest-runner].
• stick with how it already works until someone brighter than me offers a solution = )

[jaraco]

I've looked into the first two options in pytest-dev/pytest#567, but due to the implementation of distutils (which drives the processing of options to a distutils command, which pytest-runner is), there's no mechanism that I found to cleanly (a) introspect options to pytest or (b) inject them into the distutils command.

Option 3, moving to using a config file, falls prey to the same problem. There's already a mechanism for supplying options in a config file. But since pytest itself already accepts most of its options via a config file in a field called addopts, it seems prudent to match that expectation in the runner. Furthermore, once pytest starts, it already honors options provided to it in pytest.ini or tox.ini, so it doesn't make sense to have pytest-runner solicit those same values in a different file in a different format.

If someone can come up with a solution that allows the options to pytest to be passed naturally (without forcing a great deal of duplication in the runner code), I'd be all for that, but I'm not optimistic it's even possible.

That said, I've started work on a new project that aims to move away from distutils and setuptools for dependency resolution, while providing some of the same benefits that setup_requires and tests_require do, and that's rwt. With rwt, you could do something like this:

python -m rwt -r test_requirements.txt -- -m pytest --cov=plugin.pyx --cov-config=.coveragerc

So you get the elegance of passing parameters directly to pytest, and the convenience of resolving the test requirements into an ephemeral environment, and all without the trouble of setting up virtualenvs or leaving a lot of cruft around after. I do dislike that python -m rwt -r test_requirements.txt is so verbose... and the -- separating the pip args from the running process args isn't exactly beautiful. And, of course, you have to have rwt installed, which brings its own bootstrapping problems (unless it eventually gets rolled into pip, which is a possibility).

Anyway, food for thought. I hope this inspires some ideas, and I welcome suggestions and PRs, but as it stands, I don't see any way to move this forward, so I'm going to close it until such a point as it becomes actionable.

[nicoddemus]

FWIW I just opened pytest-dev/pytest#1813 which attempts to fix pytest-dev/pytest#567, so your feedback is welcome @jaraco. 😁

[kodonnell]

Thanks @jaraco - you've clearly thought this one through, and I'm happy to close it. rwt sounds nifty too - good luck.