Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Display pytest failures as GitHub annotations in GitHub Actions #3207

Merged
merged 4 commits into from
Nov 7, 2023

Conversation

echoix
Copy link
Member

@echoix echoix commented Oct 19, 2023

To help solving pytest failures currently blocking the required checks, and for general usability (without having to fully open in a IDE to navigate PRs), I suggest here to add a pytest plugin (https://github.com/pytest-dev/pytest-github-actions-annotate-failures) that bridges the pytest failures and special log format that is understood by GitHub actions (https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions).

They include a screenshot of what it could look like,
image

But here is what it looks like now, for our repo:

  1. In the log, there are additional coloured lines, but that is only a side effect image
  2. In a PR's check tab, we see in each collapsible section an indicator when there are some annotations image
    a. More specifically, this image
  3. When clicking on the check, the annotation count marker, or going to the summary tab of a GitHub Action workflow run, there is a list of annotations, errors only here (other tools can create warnings, notices and debug messages if wanted), image
  4. Clicking on an annotation brings to the code, with the line in question, here all are line 1 (why?) image
  5. I created a failure in another test run by pytest in another of my branches to illustrate another line:
    a. image
    b. When the error is in a modified file: image

@neteler neteler added this to the 8.4.0 milestone Nov 4, 2023
Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm for merging this as is. It seems faster to navigate and overall helpful especially since the alternative is going through logs.

The wrong line number for Pylint warning is slightly misleading, but the warning may be potentially still more readable for some people than just the log. It does not happen always. I wonder if the original in the log is better. That seems like a good fallback.

@echoix
Copy link
Member Author

echoix commented Nov 7, 2023

I think it's the plugin that doesn't report the line number like a real pytest test.

@wenzeslaus wenzeslaus merged commit 9defa46 into OSGeo:main Nov 7, 2023
19 checks passed
@echoix echoix deleted the Pytest-github-annot branch November 28, 2023 23:53
HuidaeCho pushed a commit to HuidaeCho/grass that referenced this pull request Jan 9, 2024
…o#3207)

Add a pytest plugin that bridges the pytest failures and special log format that is understood by GitHub Actions creating annotations in GitHub PRs. This avoids browsing the log or navigating IDE when locating the failing piece of code in question. While some Pylint-detected issues may show line 1 instead of the actual line, annotation is still helpful as a first step in learning about the detected issue.
@wenzeslaus wenzeslaus changed the title Display pytest failures as GitHub annotations in GitHub Actions CI: Display pytest failures as GitHub annotations in GitHub Actions Jun 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants