feat: improve e2e on PR process

[JorTurFer]

Signed-off-by: Jorge Turrado jorge_turrado@hotmail.es

This PR introduces 2 big changes to improve the proces:

• All PRs will be blocked till e2e tests pass or till the label ok-to-merge is manually added
• Every member of the team @kedacore/keda-e2e-test-executors will be able to trigger the e2e tests

Checklist

• Commits are signed with Developer Certificate of Origin (DCO - learn more)
• Tests have been added
• A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
• A PR is opened to update the documentation on (repo) (if applicable)
• Changelog has been updated and is aligned with our changelog requirements

Fixes #3004

[tomkerkhove]

• All PRs will be blocked till e2e tests pass or till the label ok-to-merge is manually added

I love this idea! Good thinking!

Some follow-up ideas for this/next PR:

• Would it make sense to use tests/ok-to-merge or so that we are more flexible for the future?
• Can we add a similar label for when the tests have failed?
• We might want to add a new workflow to automatically tag tests/not-run for all new PRs (happy to help here)

Thoughts?

[tomkerkhove]

LGTM, but added some suggestions in the overall comments

[tomkerkhove]

Added @v-shenoy to contributors who can run tests

[tomkerkhove]

• We might want to add a new workflow to automatically tag tests/not-run for all new PRs (happy to help here)

We can use https://github.com/marketplace/actions/labeler so that we don't have to maintain anything other than config

[tomkerkhove]

We can use https://github.com/marketplace/actions/verify-pull-request-labels to only allow to merge when the tests are NOT failing. (so ran or not ran)

[JorTurFer]

We can use github.com/marketplace/actions/verify-pull-request-labels to only allow to merge when the tests are NOT failing. (so ran or not ran)

Which is the difference with the current approach?
I mean, the already existing task works quite similar

[JorTurFer]

We can use github.com/marketplace/actions/labeler so that we don't have to maintain anything other than config

I checked this and I'm not sure if it's worth because this labels based on paths (or am I missing something?). Even if we use this task for adding labels, we need a mechanism to remove them in the case of triggering several times the e2e tests

[v-shenoy]

Love this, will make contributions easier :)

[tomkerkhove]

We can use github.com/marketplace/actions/verify-pull-request-labels to only allow to merge when the tests are NOT failing. (so ran or not ran)

Which is the difference with the current approach? I mean, the already existing task works quite similar

The difference is that I'm a moron and looked over it, haha. Sorry about that!

We can use github.com/marketplace/actions/labeler so that we don't have to maintain anything other than config

I checked this and I'm not sure if it's worth because this labels based on paths (or am I missing something?). Even if we use this task for adding labels, we need a mechanism to remove them in the case of triggering several times the e2e tests

This was to cover my suggestion to add tests/not-run label for new PRs

Love this, will make contributions easier :)

We strive to make things super simple, and there's always room for improvement. Happy to see you're liking it.

[JorTurFer]

This was to cover my suggestion to add tests/not-run label for new PRs

The problem is that once it passes the first time, the label is removed, we still need a mechanism to add the label again if we want to trigger the e2e test again. The workflow in the PR it's idempotent and you can run it several times, blocking the PR on every execution because label is added/removed on every execution

I think that this labeler action doesn't allow adding labels from a workflow with comment trigger because the parameters don't contain any reference to PR info, so I guess that it's implicitly get from the github.context

TBH, I have 0 experience with this and I'm could be missing any important point