4 tests fail on FreeBSD

[yurivict]

py-pytest-mpl-test.log

Version: 0.16.1
Python-3.9
FreeBSD 13.1

[ConorMacBride]

Thanks @yurivict for testing on FreeBSD. The pytest-mpl test suite needs a specific version of FreeType installed (currently 2.6.1) so it can render fonts consistently. I can see in the log that 2.12.1 is being used (which matches https://www.freebsd.org/cgi/ports.cgi?query=^freetype2-2.12.1_2&stype=name). If you can install 2.6.1, you should be able to get matplotlib to use that version. Alternatively, installing matplotlib with pip will install a version compiled with 2.6.1, but I guess you want to get the tests to pass using system packages only.

[yurivict]

Yes, I can't change the version of freetype because I work in the context of the FreeBSD port.

[ConorMacBride]

I see you've added a devel/py-pytest-mpl port here, thanks! The tests under tests/subtests are particularly sensitive to the FreeType and should probably all be skipped.

Although, I've just noticed another issue in the log which should be fixed first:

ValueError: The 'pytest_mpl.summary.html' package was not installed in a way that PackageLoader understands.

The patch in #180 might fix this, but I don't fully understand the root cause.

[QuLogic]

How does one update the tests here? I see that #184 added Matplotlib 3.6 hashes for subtests, but there are no hashes in tests/baseline/hashes/ for several recent Matplotlib. I would like to at least generate hashes for more recent FreeType, but I'm not sure how best to confirm them.

[yurivict]

Now many more tests fail in 0.17.0: see log.