Upload image errors to GitHub Actions

[nulano]

As GitHub Actions allows direct artifact uploads, failed image comparisons can now be uploaded directly, instead of using an external server (unlike in #2862).

The failed comparisons are saved in a randomly named directory (from tempfile.mdktemp), which is then uploaded in an artifact named errors.zip.

Test Docker is not included in this, as this is a bit more complicated there.

Sample commit with failing test: nulano@165d457
Test (Ubuntu and macOS): https://github.com/nulano/Pillow/runs/257598825
Test Windows: https://github.com/nulano/Pillow/runs/257598798

[hugovk]

Do you know if there's any file quota for the uploads, and if it's possible to delete uploads if needed?

[nulano]

Do you know if there's any file quota for the uploads, and if it's possible to delete uploads if needed?

These exact questions were asked here: actions/upload-artifact#9

Based on that discussion, it seems there is no way to delete artifacts, and that there is a bug which limits uploads to 5GB, with the total limit at over 50GB, but I can't find an official response.

According to the docs, artifacts are automatically deleted after 30 days for a PR and 90 days for a push.

[hugovk]

Let's give this a go.

However, the GHA Windows CI jobs were failing due to:

At D:\a\_temp\6e41fc6a-b1db-45df-b5a1-eac4e0dc7db0.ps1:2 char:31
+ "%pythonLocation%\python.exe" -m pip install wheel pytest pytest-cov
+                               ~~
Unexpected token '-m' in expression or statement.

I restarted it, and they all now passed, except for pypy3.6 which didn't restart.

There's now no restart button.

Please could you rebase or re-commit or something to re-trigger the CI? Then we can merge once green.

Thanks!

[radarhere]

I rebased the commit, and now all tests pass.

[hugovk]

Thank you!