Add reusable workflow for scheduled jobs to notify on slack. #3873

DimitrisJim · 2023-06-16T18:56:13Z

Description

Pending discussion on which channel to post (configurable via the WebHook) and for which workflows (hermes cron job is one, might want one for e2e-upgrade which is also on a schedule).

Current message drops a lot of information for starters, some of which might not really be needed (see a preview of it on the action repo here).

closes: #3842

Commit Message / Changelog Entry

type: commit message

see the guidelines for commit messages. (view raw markdown for examples)

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

Targeted PR against correct branch (see CONTRIBUTING.md).
Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
Code follows the module structure standards and Go style guide.
Wrote unit and integration tests.
Updated relevant documentation (docs/) or specification (x/<module>/spec/).
Added relevant godoc comments.
Provide a commit message to be used for the changelog entry in the PR description for review.
Re-reviewed Files changed in the Github PR explorer.
Review Codecov Report in the comment section below once CI passes.

DimitrisJim · 2023-07-03T08:25:55Z

.github/workflows/slack-notify.yaml

+          status: ${{ job.status }}
+          fields: repo,message,commit,author,action,eventName,ref,workflow,job,took,pullRequest
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}


added this for a private ibc-e2e channel Carlos created for now.

chatton · 2023-10-04T14:03:54Z

small update, we recently updated the upgrade workflows to run when the files/workflows related to upgrades have changed. It's probably still no harm to run them on a schedule if we want think we want to.

.github/workflows/slack-notify.yaml

chatton · 2023-11-07T10:02:09Z

.github/workflows/e2e-upgrade.yaml

+    uses: ./.github/workflows/slack-notify.yaml
+    secrets: inherit
+    # Only send a slack notification if any of the tests failed, run only on cron jobs
+    if: ${{ always() && contains(needs.*.result, 'failure') && github.event_name == 'schedule' }}


This works fine but there is a built in {{ failure() }} example.

Should we use that instead?

Also, I don't think this lets us know which one failed (I don't think that's a huge problem since we will just click on the link and see for ourselves)

This works fine but there is a built in {{ failure() }} example.

does this also apply to jobs? I remember this is an annoying discrepancy and using the contains with needs was the only workaround to getting the behaviour "fail if any of these jobs failed".

I don't think this lets us know which one failed

Yup, tried to do that first go around. Requires additional inputs to re-usable workflow and explicit passing of secrets with inherit, I didn't particularly like the way it was shaping out so went with the aggregate.

Ah I missed that it was for a step and not a job! I'm not sure if it can be used the same way. Happy to leave it for now, either way it achieves the same result.

chatton

LGTM left one suggestion, but always happy to update as needed 👍

damiannolan

Did you ever manage to get this working? what channel would it post to?

Happy to approve if its g2g! :)

damiannolan · 2024-01-24T12:55:03Z

.github/workflows/e2e-upgrade.yaml

@@ -131,3 +131,10 @@ jobs:
      relayer-type: rly
      relayer-image: ghcr.io/cosmos/relayer
      relayer-tag: latest
+
+  slack-notify:
+    needs: [upgrade-v5-hermes, upgrade-v7-hermes, upgrade-v7_1-hermes, upgrade-v8-hermes, upgrade-v5-rly, upgrade-v7-rly, upgrade-v7_1-rly, upgrade-v8-rly]


I guess this is slightly stale now!

lol. this did work but kinda lost most of its sense when we stopped nightly runs for relayer we don't use on each PR. Could still use it for upgrades if people would want it (after I freshen it up)

Yeah maybe we need to re-evaluate what we would like to run on a cron schedule and see if people think it would be valuable to have a slack notification channel for it!

I can add it to the agenda for our next sync

Running upgrade tests sounds fine. But there's not really much value in it if they're always using static image tags.
If we were running anything on a cron schedule it I think it should be at least picking up some latest code, build a fresh image and run a test harness against it.

#5762 is a good reason to add slack notify workflow in. I've rm'ed the changes to upgrade e2e file for time being, can add these at a later point.

crodriguezvega · 2024-02-19T09:19:25Z

@DimitrisJim Can we merge this one now? Or is there anything that we need to agree on or implement?

DimitrisJim · 2024-02-19T09:41:51Z

think this should be fine @crodriguezvega! just adds the workflow!

crodriguezvega

Let's merge it!

DimitrisJim force-pushed the slack-notify-workflow branch from 7585014 to 4992682 Compare June 16, 2023 18:58

DimitrisJim added the github_actions Pull requests that update Github_actions code label Jun 19, 2023

DimitrisJim commented Jul 3, 2023

View reviewed changes

chatton reviewed Oct 4, 2023

View reviewed changes

.github/workflows/slack-notify.yaml Outdated Show resolved Hide resolved

DimitrisJim force-pushed the slack-notify-workflow branch 9 times, most recently from ee1c940 to cdf9022 Compare November 2, 2023 02:18

Add reusable workflow for scheduled jobs to notify on slack.

963eb9d

DimitrisJim force-pushed the slack-notify-workflow branch 3 times, most recently from 50e504e to 00c9681 Compare November 2, 2023 11:50

DimitrisJim marked this pull request as ready for review November 2, 2023 12:22

DimitrisJim requested review from colin-axner, AdityaSripal, crodriguezvega, charleenfei and damiannolan as code owners November 2, 2023 12:22

Move job into e2e-upgrade.

2139430

DimitrisJim force-pushed the slack-notify-workflow branch from 00c9681 to 2139430 Compare November 2, 2023 12:35

chatton reviewed Nov 2, 2023

View reviewed changes

.github/workflows/slack-notify.yaml Show resolved Hide resolved

chatton reviewed Nov 7, 2023

View reviewed changes

chatton approved these changes Nov 7, 2023

View reviewed changes

DimitrisJim mentioned this pull request Nov 30, 2023

Workflow to trigger wasm E2Es with a specific tag #5288

Merged

9 tasks

crodriguezvega added priority PRs that need prompt reviews and removed priority PRs that need prompt reviews labels Jan 15, 2024

damiannolan reviewed Jan 24, 2024

View reviewed changes

DimitrisJim added 3 commits January 31, 2024 09:16

rm notification on upgrade workflow for time being.

6dfea17

Merge branch 'main' into slack-notify-workflow

3780654

Merge branch 'main' into slack-notify-workflow

94f3113

DimitrisJim and others added 2 commits February 19, 2024 11:41

Merge branch 'main' into slack-notify-workflow

f3febda

Merge branch 'main' into slack-notify-workflow

f480d84

crodriguezvega approved these changes Feb 20, 2024

View reviewed changes

DimitrisJim merged commit d266170 into main Feb 20, 2024
65 checks passed

DimitrisJim deleted the slack-notify-workflow branch February 20, 2024 10:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add reusable workflow for scheduled jobs to notify on slack. #3873

Add reusable workflow for scheduled jobs to notify on slack. #3873

DimitrisJim commented Jun 16, 2023 •

edited

Loading

DimitrisJim Jul 3, 2023 •

edited

Loading

chatton commented Oct 4, 2023

chatton Nov 7, 2023

DimitrisJim Nov 7, 2023

chatton Nov 7, 2023

chatton left a comment

damiannolan left a comment

damiannolan Jan 24, 2024

DimitrisJim Jan 24, 2024

damiannolan Jan 24, 2024

damiannolan Jan 24, 2024

DimitrisJim Jan 31, 2024

crodriguezvega commented Feb 19, 2024

DimitrisJim commented Feb 19, 2024

crodriguezvega left a comment

Add reusable workflow for scheduled jobs to notify on slack. #3873

Add reusable workflow for scheduled jobs to notify on slack. #3873

Conversation

DimitrisJim commented Jun 16, 2023 • edited Loading

Description

Commit Message / Changelog Entry

DimitrisJim Jul 3, 2023 • edited Loading

Choose a reason for hiding this comment

chatton commented Oct 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chatton left a comment

Choose a reason for hiding this comment

damiannolan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

crodriguezvega commented Feb 19, 2024

DimitrisJim commented Feb 19, 2024

crodriguezvega left a comment

Choose a reason for hiding this comment

DimitrisJim commented Jun 16, 2023 •

edited

Loading

DimitrisJim Jul 3, 2023 •

edited

Loading