#3513: Show muted alerts in the Alert Groups API #3797

grobinson-grafana · 2024-04-03T12:38:21Z

This commit updates /api/v2/alerts/groups to show if an alert is suppressed from one or more active or mute time intervals. While the muted by field can be found in /api/v2/alerts, it is not used here because /api/v2/alerts does not take aggregation or routing into consideration.

It also updates the UI to support filtering muted alerts via the Muted checkbox.

grobinson-grafana · 2024-04-05T16:21:08Z

I added a checkbox for filtering Muted alerts and a label to show if the alert is muted, like we have for silenced and inhibited alerts.

grobinson-grafana · 2024-05-13T13:38:40Z

api/v2/api.go

@@ -86,16 +88,18 @@ type (
 func NewAPI(
 	alerts provider.Alerts,
 	gf groupsFn,
-	sf getAlertStatusFn,
+	asf getAlertStatusFn,


Not the best variable name, option to suggestions! 😄

grobinson-grafana · 2024-05-13T13:39:33Z

api/v2/api.go

@@ -407,17 +411,22 @@ func (api *API) getAlertGroupsHandler(params alertgroup_ops.GetAlertGroupsParams
 	res := make(open_api_models.AlertGroups, 0, len(alertGroups))

 	for _, alertGroup := range alertGroups {
+		mutedBy, isMuted := api.groupMutedFunc(alertGroup.RouteID, alertGroup.GroupKey)
+		if !*params.Muted && isMuted {


This skips muted groups if muted is false.

grobinson-grafana · 2024-05-13T13:43:15Z

dispatch/dispatch.go

@@ -206,6 +206,8 @@ type AlertGroup struct {
 	Alerts   types.AlertSlice
 	Labels   model.LabelSet
 	Receiver string
+	GroupKey string
+	RouteID  string


Needed to add RouteID here because we added it to the marker. Without it, we cannot see if a group is muted or not.

This commit updates /api/v2/alerts/groups to show if an alert is suppressed from one or more active or mute time intervals. While the muted by field can be found in /api/v2/alerts, it is not used here because /api/v2/alerts does not take aggregation or routing into consideration. It also updates the UI to support filtering muted alerts via the Muted checkbox. Signed-off-by: George Robinson <george.robinson@grafana.com>

grobinson-grafana · 2024-06-22T17:52:14Z

api/v2/compat.go

@@ -117,7 +117,7 @@ func PostableSilenceToProto(s *open_api_models.PostableSilence) (*silencepb.Sile
 }

 // AlertToOpenAPIAlert converts internal alerts, alert types, and receivers to *open_api_models.GettableAlert.
-func AlertToOpenAPIAlert(alert *types.Alert, status types.AlertStatus, receivers []string) *open_api_models.GettableAlert {
+func AlertToOpenAPIAlert(alert *types.Alert, status types.AlertStatus, receivers, mutedBy []string) *open_api_models.GettableAlert {


mutedBy is separate from types.AlertStatus because it is related to groups, and does not come from the AlertMarker interface.

SuperQ · 2024-10-17T09:02:59Z

api/v2/api.go

+	asf getAlertStatusFn,
+	gmf groupMutedFunc,


How about this for consistent naming:

Suggested change

asf getAlertStatusFn,

gmf groupMutedFunc,

asf alertStatusFn,

gmf groupMutedFn,

I don't mind the types as these two are consistent:

getAlertStatusFn func(prometheus_model.Fingerprint) types.AlertStatus setAlertStatusFn func(prometheus_model.LabelSet)

It was more the variable names asf and gmf that I didn't like.

SuperQ

Minor suggestion on the function naming, otherwise LGTM.

* Release v0.28.0-rc.0 * [CHANGE] Templating errors in the SNS integration now return an error. #3531 #3879 * [FEATURE] Add a new Microsoft Teams integration based on Flows #4024 * [FEATURE] Add a new Rocket.Chat integration #3600 * [FEATURE] Add a new Jira integration #3590 #3931 * [FEATURE] Add support for `GOMEMLIMIT`, enable it via the feature flag `--enable-feature=auto-gomemlimit`. #3895 * [FEATURE] Add support for `GOMAXPROCS`, enable it via the feature flag `--enable-feature=auto-gomaxprocs`. #3837 * [FEATURE] Add support for limits of silences including the maximum number of active and pending silences, and the maximum size per silence (in bytes). You can use the flags `--silences.max-silences` and `--silences.max-silence-size-bytes` to set them accordingly #3852 #3862 #3866 #3885 #3886 #3877 * [FEATURE] Muted alerts now show whether they are suppressed or not in both the `/api/v2/alerts` endpoint and the Alertmanager UI. #3793 #3797 #3792 * [ENHANCEMENT] Add support for `content`, `username` and `avatar_url` in the Discord integration. `content` and `username` also support templating. #4007 * [ENHANCEMENT] Only invalidate the silences cache if a new silence is created or an existing silence replaced - should improve latency on both `GET api/v2/alerts` and `POST api/v2/alerts` API endpoint. #3961 * [ENHANCEMENT] Add image source label to Dockerfile. To get changelogs shown when using Renovate #4062 * [ENHANCEMENT] Build using go 1.23 #4071 * [ENHANCEMENT] Support setting a global SMTP TLS configuration. #3732 * [ENHANCEMENT] The setting `room_id` in the WebEx integration can now be templated to allow for dynamic room IDs. #3801 * [ENHANCEMENT] Enable setting `message_thread_id` for the Telegram integration. #3638 * [ENHANCEMENT] Support the `since` and `humanizeDuration` functions to templates. This means users can now format time to more human-readable text. #3863 * [ENHANCEMENT] Support the `date` and `tz` functions to templates. This means users can now format time in a specified format and also change the timezone to their specific locale. #3812 * [ENHANCEMENT] Latency metrics now support native histograms. #3737 * [BUGFIX] Fix the SMTP integration not correctly closing an SMTP submission, which may lead to unsuccessful dispatches being marked as successful. #4006 * [BUGFIX] The `ParseMode` option is now set explicitly in the Telegram integration. If we don't HTML tags had not been parsed by default. #4027 * [BUGFIX] Fix a memory leak that was caused by updates silences continuously. #3930 * [BUGFIX] Fix hiding secret URLs when the URL is incorrect. #3887 * [BUGFIX] Fix a race condition in the alerts - it was more of a hypothetical race condition that could have occurred in the alert reception pipeline. #3648 * [BUGFIX] Fix a race condition in the alert delivery pipeline that would cause a firing alert that was delivered earlier to be deleted from the aggregation group when instead it should have been delivered again. #3826 * [BUGFIX] Fix version in APIv1 deprecation notice. #3815 * [BUGFIX] Fix crash errors when using `url_file` in the Webhook integration. #3800 * [BUGFIX] fix `Route.ID()` returns conflicting IDs. #3803 * [BUGFIX] Fix deadlock on the alerts memory store. #3715 * [BUGFIX] Fix `amtool template render` when using the default values. #3725 * [BUGFIX] Fix `webhook_url_file` for both the Discord and Microsoft Teams integrations. #3728 #3745 --------- Signed-off-by: SuperQ <superq@gmail.com> Signed-off-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gotjosh <josue.abreu@gmail.com>

* [CHANGE] Templating errors in the SNS integration now return an error. #3531 #3879 * [CHANGE] Adopt log/slog, drop go-kit/log #4089 * [FEATURE] Add a new Microsoft Teams integration based on Flows #4024 * [FEATURE] Add a new Rocket.Chat integration #3600 * [FEATURE] Add a new Jira integration #3590 #3931 * [FEATURE] Add support for `GOMEMLIMIT`, enable it via the feature flag `--enable-feature=auto-gomemlimit`. #3895 * [FEATURE] Add support for `GOMAXPROCS`, enable it via the feature flag `--enable-feature=auto-gomaxprocs`. #3837 * [FEATURE] Add support for limits of silences including the maximum number of active and pending silences, and the maximum size per silence (in bytes). You can use the flags `--silences.max-silences` and `--silences.max-silence-size-bytes` to set them accordingly #3852 #3862 #3866 #3885 #3886 #3877 * [FEATURE] Muted alerts now show whether they are suppressed or not in both the `/api/v2/alerts` endpoint and the Alertmanager UI. #3793 #3797 #3792 * [ENHANCEMENT] Add support for `content`, `username` and `avatar_url` in the Discord integration. `content` and `username` also support templating. #4007 * [ENHANCEMENT] Only invalidate the silences cache if a new silence is created or an existing silence replaced - should improve latency on both `GET api/v2/alerts` and `POST api/v2/alerts` API endpoint. #3961 * [ENHANCEMENT] Add image source label to Dockerfile. To get changelogs shown when using Renovate #4062 * [ENHANCEMENT] Build using go 1.23 #4071 * [ENHANCEMENT] Support setting a global SMTP TLS configuration. #3732 * [ENHANCEMENT] The setting `room_id` in the WebEx integration can now be templated to allow for dynamic room IDs. #3801 * [ENHANCEMENT] Enable setting `message_thread_id` for the Telegram integration. #3638 * [ENHANCEMENT] Support the `since` and `humanizeDuration` functions to templates. This means users can now format time to more human-readable text. #3863 * [ENHANCEMENT] Support the `date` and `tz` functions to templates. This means users can now format time in a specified format and also change the timezone to their specific locale. #3812 * [ENHANCEMENT] Latency metrics now support native histograms. #3737 * [ENHANCEMENT] Add timeout option for webhook notifier. #4137 * [BUGFIX] Fix the SMTP integration not correctly closing an SMTP submission, which may lead to unsuccessful dispatches being marked as successful. #4006 * [BUGFIX] The `ParseMode` option is now set explicitly in the Telegram integration. If we don't HTML tags had not been parsed by default. #4027 * [BUGFIX] Fix a memory leak that was caused by updates silences continuously. #3930 * [BUGFIX] Fix hiding secret URLs when the URL is incorrect. #3887 * [BUGFIX] Fix a race condition in the alerts - it was more of a hypothetical race condition that could have occurred in the alert reception pipeline. #3648 * [BUGFIX] Fix a race condition in the alert delivery pipeline that would cause a firing alert that was delivered earlier to be deleted from the aggregation group when instead it should have been delivered again. #3826 * [BUGFIX] Fix version in APIv1 deprecation notice. #3815 * [BUGFIX] Fix crash errors when using `url_file` in the Webhook integration. #3800 * [BUGFIX] fix `Route.ID()` returns conflicting IDs. #3803 * [BUGFIX] Fix deadlock on the alerts memory store. #3715 * [BUGFIX] Fix `amtool template render` when using the default values. #3725 * [BUGFIX] Fix `webhook_url_file` for both the Discord and Microsoft Teams integrations. #3728 #3745 * [BUGFIX] Fix wechat api link #4084 * [BUGFIX] Fix build info metric #4166 Signed-off-by: SuperQ <superq@gmail.com>

* [CHANGE] Templating errors in the SNS integration now return an error. #3531 #3879 * [CHANGE] Adopt log/slog, drop go-kit/log #4089 * [FEATURE] Add a new Microsoft Teams integration based on Flows #4024 * [FEATURE] Add a new Rocket.Chat integration #3600 * [FEATURE] Add a new Jira integration #3590 #3931 * [FEATURE] Add support for `GOMEMLIMIT`, enable it via the feature flag `--enable-feature=auto-gomemlimit`. #3895 * [FEATURE] Add support for `GOMAXPROCS`, enable it via the feature flag `--enable-feature=auto-gomaxprocs`. #3837 * [FEATURE] Add support for limits of silences including the maximum number of active and pending silences, and the maximum size per silence (in bytes). You can use the flags `--silences.max-silences` and `--silences.max-silence-size-bytes` to set them accordingly #3852 #3862 #3866 #3885 #3886 #3877 * [FEATURE] Muted alerts now show whether they are suppressed or not in both the `/api/v2/alerts` endpoint and the Alertmanager UI. #3793 #3797 #3792 * [ENHANCEMENT] Add support for `content`, `username` and `avatar_url` in the Discord integration. `content` and `username` also support templating. #4007 * [ENHANCEMENT] Only invalidate the silences cache if a new silence is created or an existing silence replaced - should improve latency on both `GET api/v2/alerts` and `POST api/v2/alerts` API endpoint. #3961 * [ENHANCEMENT] Add image source label to Dockerfile. To get changelogs shown when using Renovate #4062 * [ENHANCEMENT] Build using go 1.23 #4071 * [ENHANCEMENT] Support setting a global SMTP TLS configuration. #3732 * [ENHANCEMENT] The setting `room_id` in the WebEx integration can now be templated to allow for dynamic room IDs. #3801 * [ENHANCEMENT] Enable setting `message_thread_id` for the Telegram integration. #3638 * [ENHANCEMENT] Support the `since` and `humanizeDuration` functions to templates. This means users can now format time to more human-readable text. #3863 * [ENHANCEMENT] Support the `date` and `tz` functions to templates. This means users can now format time in a specified format and also change the timezone to their specific locale. #3812 * [ENHANCEMENT] Latency metrics now support native histograms. #3737 * [ENHANCEMENT] Add full width to adaptive card for msteamsv2 #4135 * [ENHANCEMENT] Add timeout option for webhook notifier. #4137 * [ENHANCEMENT] Update config to allow showing secret values when marshaled #4158 * [ENHANCEMENT] Enable templating for Jira project and issue_type #4159 * [BUGFIX] Fix the SMTP integration not correctly closing an SMTP submission, which may lead to unsuccessful dispatches being marked as successful. #4006 * [BUGFIX] The `ParseMode` option is now set explicitly in the Telegram integration. If we don't HTML tags had not been parsed by default. #4027 * [BUGFIX] Fix a memory leak that was caused by updates silences continuously. #3930 * [BUGFIX] Fix hiding secret URLs when the URL is incorrect. #3887 * [BUGFIX] Fix a race condition in the alerts - it was more of a hypothetical race condition that could have occurred in the alert reception pipeline. #3648 * [BUGFIX] Fix a race condition in the alert delivery pipeline that would cause a firing alert that was delivered earlier to be deleted from the aggregation group when instead it should have been delivered again. #3826 * [BUGFIX] Fix version in APIv1 deprecation notice. #3815 * [BUGFIX] Fix crash errors when using `url_file` in the Webhook integration. #3800 * [BUGFIX] fix `Route.ID()` returns conflicting IDs. #3803 * [BUGFIX] Fix deadlock on the alerts memory store. #3715 * [BUGFIX] Fix `amtool template render` when using the default values. #3725 * [BUGFIX] Fix `webhook_url_file` for both the Discord and Microsoft Teams integrations. #3728 #3745 * [BUGFIX] Fix wechat api link #4084 * [BUGFIX] Fix build info metric #4166 * [BUGFIX] Fix UTF-8 not allowed in Equal field for inhibition rules #4177 Signed-off-by: SuperQ <superq@gmail.com>

* [CHANGE] Templating errors in the SNS integration now return an error. prometheus#3531 prometheus#3879 * [CHANGE] Adopt log/slog, drop go-kit/log prometheus#4089 * [FEATURE] Add a new Microsoft Teams integration based on Flows prometheus#4024 * [FEATURE] Add a new Rocket.Chat integration prometheus#3600 * [FEATURE] Add a new Jira integration prometheus#3590 prometheus#3931 * [FEATURE] Add support for `GOMEMLIMIT`, enable it via the feature flag `--enable-feature=auto-gomemlimit`. prometheus#3895 * [FEATURE] Add support for `GOMAXPROCS`, enable it via the feature flag `--enable-feature=auto-gomaxprocs`. prometheus#3837 * [FEATURE] Add support for limits of silences including the maximum number of active and pending silences, and the maximum size per silence (in bytes). You can use the flags `--silences.max-silences` and `--silences.max-silence-size-bytes` to set them accordingly prometheus#3852 prometheus#3862 prometheus#3866 prometheus#3885 prometheus#3886 prometheus#3877 * [FEATURE] Muted alerts now show whether they are suppressed or not in both the `/api/v2/alerts` endpoint and the Alertmanager UI. prometheus#3793 prometheus#3797 prometheus#3792 * [ENHANCEMENT] Add support for `content`, `username` and `avatar_url` in the Discord integration. `content` and `username` also support templating. prometheus#4007 * [ENHANCEMENT] Only invalidate the silences cache if a new silence is created or an existing silence replaced - should improve latency on both `GET api/v2/alerts` and `POST api/v2/alerts` API endpoint. prometheus#3961 * [ENHANCEMENT] Add image source label to Dockerfile. To get changelogs shown when using Renovate prometheus#4062 * [ENHANCEMENT] Build using go 1.23 prometheus#4071 * [ENHANCEMENT] Support setting a global SMTP TLS configuration. prometheus#3732 * [ENHANCEMENT] The setting `room_id` in the WebEx integration can now be templated to allow for dynamic room IDs. prometheus#3801 * [ENHANCEMENT] Enable setting `message_thread_id` for the Telegram integration. prometheus#3638 * [ENHANCEMENT] Support the `since` and `humanizeDuration` functions to templates. This means users can now format time to more human-readable text. prometheus#3863 * [ENHANCEMENT] Support the `date` and `tz` functions to templates. This means users can now format time in a specified format and also change the timezone to their specific locale. prometheus#3812 * [ENHANCEMENT] Latency metrics now support native histograms. prometheus#3737 * [ENHANCEMENT] Add full width to adaptive card for msteamsv2 prometheus#4135 * [ENHANCEMENT] Add timeout option for webhook notifier. prometheus#4137 * [ENHANCEMENT] Update config to allow showing secret values when marshaled prometheus#4158 * [ENHANCEMENT] Enable templating for Jira project and issue_type prometheus#4159 * [BUGFIX] Fix the SMTP integration not correctly closing an SMTP submission, which may lead to unsuccessful dispatches being marked as successful. prometheus#4006 * [BUGFIX] The `ParseMode` option is now set explicitly in the Telegram integration. If we don't HTML tags had not been parsed by default. prometheus#4027 * [BUGFIX] Fix a memory leak that was caused by updates silences continuously. prometheus#3930 * [BUGFIX] Fix hiding secret URLs when the URL is incorrect. prometheus#3887 * [BUGFIX] Fix a race condition in the alerts - it was more of a hypothetical race condition that could have occurred in the alert reception pipeline. prometheus#3648 * [BUGFIX] Fix a race condition in the alert delivery pipeline that would cause a firing alert that was delivered earlier to be deleted from the aggregation group when instead it should have been delivered again. prometheus#3826 * [BUGFIX] Fix version in APIv1 deprecation notice. prometheus#3815 * [BUGFIX] Fix crash errors when using `url_file` in the Webhook integration. prometheus#3800 * [BUGFIX] fix `Route.ID()` returns conflicting IDs. prometheus#3803 * [BUGFIX] Fix deadlock on the alerts memory store. prometheus#3715 * [BUGFIX] Fix `amtool template render` when using the default values. prometheus#3725 * [BUGFIX] Fix `webhook_url_file` for both the Discord and Microsoft Teams integrations. prometheus#3728 prometheus#3745 * [BUGFIX] Fix wechat api link prometheus#4084 * [BUGFIX] Fix build info metric prometheus#4166 * [BUGFIX] Fix UTF-8 not allowed in Equal field for inhibition rules prometheus#4177 Signed-off-by: SuperQ <superq@gmail.com> Signed-off-by: heartwilltell <heartwilltell@gmail.com>

grobinson-grafana marked this pull request as draft April 3, 2024 12:38

grobinson-grafana mentioned this pull request Apr 3, 2024

Muted alerts are not suppressed in API #3513

Closed

grobinson-grafana changed the title ~~Show muted alerts in the Alert Groups API~~ #3513: Show muted alerts in the Alert Groups API Apr 3, 2024

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 5c4b023 to ce8652d Compare April 3, 2024 14:52

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 3de4eba to 08836ab Compare April 8, 2024 09:53

grobinson-grafana marked this pull request as ready for review April 8, 2024 09:54

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 08836ab to 681f2a7 Compare April 11, 2024 14:51

grobinson-grafana marked this pull request as draft April 11, 2024 15:01

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 681f2a7 to 1d227f1 Compare April 29, 2024 17:07

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 0c1a5f8 to 0cd1216 Compare May 13, 2024 13:18

grobinson-grafana commented May 13, 2024

View reviewed changes

grobinson-grafana marked this pull request as ready for review May 13, 2024 14:12

grobinson-grafana force-pushed the grobinson/mark-muted-alerts-4 branch from 7b804a1 to 6c2a305 Compare June 22, 2024 17:50

grobinson-grafana commented Jun 22, 2024

View reviewed changes

SuperQ requested review from gotjosh and SuperQ October 16, 2024 21:54

grobinson-grafana mentioned this pull request Oct 17, 2024

Release v0.28.0-rc.0 #4072

Merged

SuperQ reviewed Oct 17, 2024

View reviewed changes

SuperQ approved these changes Oct 17, 2024

View reviewed changes

gotjosh merged commit 5979dff into prometheus:main Oct 23, 2024
11 checks passed

gillesdemey mentioned this pull request Oct 25, 2024

Alerting: Alertmanager 0.28.0 support grafana/grafana#95393

Open

8 tasks

SuperQ mentioned this pull request Dec 19, 2024

Release v0.28.0 #4176

Merged

xkilian mentioned this pull request Jan 16, 2025

Support filtering muted alertes now available in Alertmanager 0.28 prymitive/karma#6401

Open

gigermocas mentioned this pull request Feb 4, 2025

Update alertmanager from 0.27.0 to 0.28.0 gigermocas/prometheus-rpm#8

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#3513: Show muted alerts in the Alert Groups API #3797

#3513: Show muted alerts in the Alert Groups API #3797

grobinson-grafana commented Apr 3, 2024 •

edited

Loading

grobinson-grafana commented Apr 5, 2024

grobinson-grafana May 13, 2024

grobinson-grafana May 13, 2024

grobinson-grafana May 13, 2024

grobinson-grafana Jun 22, 2024

SuperQ Oct 17, 2024 •

edited

Loading

grobinson-grafana Oct 23, 2024

SuperQ left a comment

#3513: Show muted alerts in the Alert Groups API #3797

#3513: Show muted alerts in the Alert Groups API #3797

Conversation

grobinson-grafana commented Apr 3, 2024 • edited Loading

grobinson-grafana commented Apr 5, 2024

grobinson-grafana May 13, 2024

Choose a reason for hiding this comment

grobinson-grafana May 13, 2024

Choose a reason for hiding this comment

grobinson-grafana May 13, 2024

Choose a reason for hiding this comment

grobinson-grafana Jun 22, 2024

Choose a reason for hiding this comment

SuperQ Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

grobinson-grafana Oct 23, 2024

Choose a reason for hiding this comment

SuperQ left a comment

Choose a reason for hiding this comment

grobinson-grafana commented Apr 3, 2024 •

edited

Loading

SuperQ Oct 17, 2024 •

edited

Loading