Tests do not seem to run in Windows in GitHub Actions

[phoebe-p]

Recent changes to the behaviour of pip and rtools meant that all regularly scheduled tests were failing (see e.g. here). To summarize:

• unclear on exactly what happens, but presumably due to changes in how pip treats packages which do not have a pyproject.toml (see here, the S4 build in the Ubuntu and MacOS workflows was failing. This was fixed by simply installing wheel before trying to build S4. All tests now seem to run and pass fine on MacOS and Ubuntu.
• S4 is not installed in Windows, so that build was failing for a different reason. The process seems to have broken when choco install started installing RTools 4.3 instead of RTools 4.0 (https://community.chocolatey.org/packages/rtools#versionhistory). This means the version of gcc/gfortran changed from 10.3.0 to 11.2.0, and for some reason the build fails. This is "fixed" by forcibly installing RTools 4.0 (Implement heterojunction in depletion approximation and fix build issues #256).

All the tests are now seemingly passing, however, it does not seem like the tests on Windows are actually running; everything seems fine, but no tests run and no coverage report is generated (see here). I am not sure how to fix this because I am not sure which package is causing it (devpy? pytest?).

@dalonsoa @Abelarm do you have any thoughts on what might be causing this?

[Abelarm]

Hi guys!!

I think the problems are the tests itself.
The first thing I would do is try to run the test locally (on some windows machine you have around) and see what's going on.
Because I remember that when I was building the pipeline for windows I could not run some example's notebooks because they kind of get "stuck" so I think is the same problem and the only way to debug it is to run it locally.

Sadly I don't have a windows pc, you cannot even run windows containers in docker without a windows host...

[Abelarm]

Hi @phoebe-p did you had a chance to look at it? Because I found a windows PC and I was able to run the test locally (without pdd) with no problem...
So the problem is on the test pipeline not the test themself...
What do you think?

[phoebe-p]

Yes, I had the same experience but forgot to report back here… I have tried a few different things, like calling pytest directly, but in that case it does not seem to find the built version of Solcore and complains the PDD is not installed. I’m not sure what else to try…

[Abelarm]

Could it be the -n "auto" the xdist package that distribute the tests?
Did you already try to remove it?

[Abelarm]

Any news? 🤔

[phoebe-p]

thanks for reminding me @Abelarm , I just tried and it seems this work for the tests (see the latest GitHub Actions runs - one of the Windows tests failed randomly because of a worker crashing, but that just seems to happen sometimes...)! For test_examples, I also removed the -n auto part for Windows but it still seems to do nothing. No idea why but I guess it is not a huge issue if the examples only run on Ubuntu and MacOS, as long as we are running the actual tests on Windows to see if the installation works.

[Abelarm]

always happy to help!

I think windows does not like notebooks or whatever.
The mysterious ways of windows...

[phoebe-p]

I will just remove the section for running the examples in Windows, since leaving it there when it does nothing will be confusing in the future. Thanks for the help again!