Pytest plugin: xdist-compatible stats collection breaks strict XML report validation

[danilomendesdias]

As documented in https://docs.pytest.org/en/latest/usage.html#record-property, record_property (or its equivalent usage through custom markers, which I think that it is the approach chosen by hypothesis) is an experimental fixture that can break if <properties> inside <testcase> is not expected by XML schema verification.

My use case is to run in a CI server with jenkins + junit plugin (which I can't modify basically anything) and I'm not able to run newer versions of hypothesis due this. I know that having hypothesis information in reports is a nice feature, but as it can cause this kind of problems, my suggestion is to make it at least configurable by adding something like --turn-off-hypothesis-stats-on-reports.

Maybe this option is already available, but I couldn't find anything on docs. I can also contribute with a PR if you guys approve the idea and point me where to start.

Thanks in advance.

[Zac-HD]

Hmm, we'll need to think about this - I added record_property because it's the only way I could find to pass stats between processes using pytest-xdist. I agree that it's not good to have this effect on XML reports, but that might be more of a Pytest issue than a Hypothesis issue...

Until we get the design sorted out, you can use pytest -p no:hypothesis to disable the plugin entirely.

[Zac-HD]

Closing this in favor of the upstream issue in Pytest. Thanks @danilomendesdias!