Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest: Enhance logging output with colors and summary #3629

Merged
merged 9 commits into from
Apr 29, 2024

Conversation

echoix
Copy link
Member

@echoix echoix commented Apr 19, 2024

To help read CI logs more easily, enabled colored output where possible (in CI only, for pytest workflow).

For pytest itself:

  • added verbose output so we can see what test is running
  • added colors so error results are WAY easier to read through (it even has syntax highlighting for python code)
  • added a short summary of passed/skipped/failed/xfailed tests
  • added the display of top durations for the test or the setup (I finally chose to not limit the number of items to output, but to raise the minimum duration for that list to ignore all durations under 0.5 sec instead of the default of 0.005s. 0.5s seems reasonable for what a slow unit test should be in my opinion. We have setup and test steps that take over 9 seconds as we can see from that)
Images

Colored pip with cache misses:
image

For errors/warnings, some I intentionnaly added to see how it looked
Colored gcc:
image

Colored pytest:
image
image
image
image
image

@github-actions github-actions bot added the CI Continuous integration label Apr 19, 2024
@echoix
Copy link
Member Author

echoix commented Apr 19, 2024

I finally removed the GCC colors since the builds fails with it, and don't fail if I remove it afterwards. I don't understand why, but since they are not super obvious, I'm kicking it out of the scope.

@echoix
Copy link
Member Author

echoix commented Apr 19, 2024

This is a side PR from what I was working on simultaneously: adding ccache or sccache, whichever was better. For now, from the numbers I have, it isn't worth it for pytest, but for windows, it might be worth it:

  • On cache hit: I have 95 seconds saved with ccache (8min 33 to 8min37sec vs 10min12sec before)
  • When building a cache: time increased to 12min39sec vs 10min12 before (2min27 more)

Both can take 6 seconds before and after to push pull the cache.

Cache from main can be used in PRs (in theory), but cache from PRs are scoped to the PR (ie: a PR can fall back to main's cache if it doesn't exist).

I'm not decided yet and didn't manage to make sccache work, as it originally had better support on windows.

@echoix echoix added this to the 8.4.0 milestone Apr 23, 2024
@echoix echoix merged commit 4b1a757 into OSGeo:main Apr 29, 2024
27 checks passed
@echoix echoix deleted the pytest-output-improvements branch May 4, 2024 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants