Change "/run-e2e" to use GitHub's Checks API

[tomkerkhove]

Change "/run-e2e" to use GitHub's Checks API instead of emojis so that PRs are blocked in case of failed tests.

• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks
• https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api

There seem to be some GitHub Actions in the marketplace, for example this one: https://github.com/marketplace/actions/github-checks

[JorTurFer]

I have been checking GitHub Checks API and I think that it doesn't match our requirements. This API allows dynamic checks creation with custom messages, reports, etc... the problem is that those checks are not considered part of the PR checks, I mean, we cannot block the PR with it.

Maybe I have missed something, but I think that GH is not ready for this... 😢

I have another option that could work, the workflow could add a label to the PR, and we could use that label to block the PR, I mean, the e2e workflow could add/remove a label in the PR with for example e2e failed (and another informative like e2e passed ) and we could add other check that monitors that label. With this approach, we could block the PR.
WDYT @tomkerkhove, @zroubalik ?

[tomkerkhove]

the problem is that those checks are not considered part of the PR checks, I mean, we cannot block the PR with it.

Normally you can link them to the PR as per the docs above. Did you give this a try?

The alternative is another approach but feels a bit like a hack, ideally we rely on checks IMO. If we link the running workflow job to the PR it should also work imo.

[JorTurFer]

Maybe I miss the part of the PR... Could you send me that link?

[tomkerkhove]

It is in the issue above: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks

[JorTurFer]

The closest thing that I found yesterday was this.
We can create a check for a specific commit hash, but despite that also having the commit is in the PR, the check is not attached to the PR checks.

Anyway, I'll take another look

[tomkerkhove]

Creates a new check run for a specific commit in a repository. Your GitHub App must have the checks:write permission to create check runs.

Was this the case?

Let me ask around

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.