[APM] Threshold alerts #59566

dgieselaar · 2020-03-06T16:52:16Z

Closes #49038 (~~actually, does it? should we remove the watcher functionality in this PR as well~~ we'll keep the Watcher one as well for the time being).

Out of scope:

header redesign (open issue [APM] Consistent Observability header and navigation redesign #40236). Added a menu next to the integrations dropdown for now.
annotations in charts (needs the event log to be queryable) [APM] Display threshold alert annotations on time-series graphs #60373
dropdown in chart (could be added, LMK if important enough)

I've not yet tested all scenarios, but it would be helpful to get some feedback while I'm out on spacetime so I can pick it up directly after.

Related issues:

Filtering the alert list on APM alerts: Ability to filter alerts management list via the URL (by alert type, tags, consumer, etc) #59545
Error handling in the alert form: Error experience in create alert flow #58181
System tags: Request for alerting internal tags structure #58417
Event log: [alerting event log] add query support #55633

Relevant changes for @elastic/kibana-alerting-services are mostly in a77f088bd5aec73716cb68a0fb9ceaeb19fa9adc and 73d6bdc5b111c61038e06e488ef37212c55a74bc.

sorenlouv · 2020-03-10T22:29:23Z

x-pack/legacy/plugins/apm/index.ts

+              'save',
+              'alerting:show',
+              'actions:show',
+              'alerting:save',


Is it possible to grant users granular access, like alerting:show bu not actions:show?

Possible for us? Or for admins?

.../plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/AlertingFlyout/index.tsx

dgieselaar · 2020-03-10T22:36:47Z

I don't think so. Previously (IIRC) there were instance specific properties (setFlyoutVisible etc) passed to the context component but they were recently moved to the form, so might be fine as a root component now. Op di 10 mrt. 2020 23:30 schreef Søren Louv-Jansen <notifications@github.com

…

: ***@***.**** commented on this pull request. ------------------------------ In x-pack/legacy/plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/AlertingFlyout/index.tsx <#59566 (comment)>: > + +export function AlertingFlyout(props: Props) { + const { addFlyoutVisible, setAddFlyoutVisibility, alertType } = props; + + const plugin = useApmPluginContext(); + + return ( + <AlertsContextProvider + value={{ + http: plugin.core.http, + toastNotifications: plugin.core.notifications.toasts, + actionTypeRegistry: + plugin.plugins.triggers_actions_ui.actionTypeRegistry, + alertTypeRegistry: plugin.plugins.triggers_actions_ui.alertTypeRegistry + }} + > is it overkill adding AlertsContextProvider at the application root? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#59566?email_source=notifications&email_token=AACWDXAPE5MGLEZRWXKJHADRG25Q7A5CNFSM4LDD7UK2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCYY5ULQ#pullrequestreview-372365870>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACWDXHSABERLQPJLX7BHJDRG25Q7ANCNFSM4LDD7UKQ> .

sorenlouv · 2020-03-12T12:28:44Z

x-pack/legacy/plugins/apm/index.ts

@@ -88,22 +88,31 @@ export const apm: LegacyPluginInitializer = kibana => {
        catalogue: ['apm'],
        privileges: {
          all: {
-            api: ['apm', 'apm_write'],
+            api: ['apm', 'apm_write', 'actions-read', 'alerting-read'],


apm_write uses snake_case. Should we stay consistent?

Btw. Should apm be renamed to apm_read?

ah.. actions-read is what's defined in the Action app endpoints...
My mind is a little hazy on this mapping concept. Are we mapping tags to privileges here? How does it work?

I understood it last time I looked into it, but that was last year. It seems this mapping concept is not immediately intuitive so adding some comments (albeit a bit verbose) might be a good idea.

Would just linking to x-pack/plugins/features/common/feature_kibana_privileges.ts where it's explained be OK? Rather than copying documentation between files?

sorenlouv · 2020-03-12T12:41:11Z

.../plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/AlertingFlyout/index.tsx

+  AlertAdd
+} from '../../../../../../../../../plugins/triggers_actions_ui/public';
+
+type AlertAddProps = React.ComponentProps<typeof AlertAdd>;


This is such an underrated ts helper

sorenlouv · 2020-03-12T12:43:16Z

x-pack/legacy/plugins/apm/public/components/shared/ServiceAlertTrigger/DurationField/index.tsx

+    parsedInterval = parseEsInterval(duration);
+  } catch (err) {
+    //
+  }


Should the error be handled somehow?

It shouldn't, but I'll leave a comment there with the reason.

In that case this feels like something that shouldn't be thrown from parseEsInterval. But I can see how that is not easily changed.

sorenlouv · 2020-03-12T12:45:03Z

x-pack/legacy/plugins/apm/public/components/shared/ServiceAlertTrigger/DurationField/index.tsx

+          value={parsedInterval?.value ?? ''}
+          onChange={e => onChange(`${e.target.value}${displayedUnit.current}`)}
+          prepend={i18n.translate(
+            'xpack.apm.serviceAlertTrigger.durationField..last',


This key looks off durationField..last

sorenlouv · 2020-03-12T12:47:59Z

x-pack/legacy/plugins/apm/public/components/shared/ServiceAlertTrigger/DurationField/index.tsx

+    //
+  }
+
+  const displayedUnit = useRef(parsedInterval?.unit);


Why is the ref needed?

To allow the user to clear the number input. I'll add a comment there as well:

// we use a ref that we only update when we've succesfully parsed // the interval. this allows the user to clear the number input // without us having to fill it with a 0 in order for it to be // succesfully parsed.

Why do we have to fill it with a 0? Is "" (empty string) not valid?

That's what this is (was) about - supporting anything that is not a valid number, which is e.g. what happens when the user clears the field. Once that happens we can no longer successfully parse the interval.

In any case I'm now using the ForLastExpression that the triggers_actions_ui plugin provides.

sorenlouv · 2020-03-12T12:50:35Z

x-pack/legacy/plugins/apm/public/components/shared/ServiceAlertTrigger/index.tsx

+    });
+
+    // eslint-disable-next-line react-hooks/exhaustive-deps
+  }, []);


😬I'll add a comment.

sorenlouv · 2020-03-12T14:08:57Z

x-pack/legacy/plugins/apm/public/components/shared/ServiceAlertTrigger/index.tsx

+      `service.name:${params.serviceName}`,
+      `transaction.type:${params.transactionType}`


I think we should use the ecs constants:

Suggested change

`service.name:${params.serviceName}`,

`transaction.type:${params.transactionType}`

`${SERVICE_NAME}:${params.serviceName}`,

`${TRANSACTION_TYPE}:${params.transactionType}`

Ntw. what do you think about renaming elasticsearch_fieldnames.ts to ecs.ts ?

It's not all ECS (unfortunately). ie, service.environment is not in the ECS specification. Same goes for error.grouping_key and a couple of others. So maybe elasticsearch_fieldnames is good enough for now?

btw, I'm removing the service.name and transaction.type tags. We're not doing anything wiht lookups now, and in the future (IIRC) we can use the params.

formgeist · 2020-03-12T14:24:43Z

@dgieselaar I've collected some immediate design feedback and polish comments;

Can we add some separation between these two form fields? 8px like the others?

I suppose we need the transaction.type selection and tag is needed to indicate the type. Wonder if we need to make this part of the trigger definition and prefill with the selected type from the Transaction page. Secondly, is the transaction.type needed for error rate alerts?

transaction.type tag shows for Error rate alerts.

Do we control the copy in these tooltips?

Add a spacer between the trigger definitions and the alert actions. Probably a EuiSpacer size: m.

Noticed that the Transaction duration alert has the type Error rate.

Another spacing form fields in the Error rate flyout

Also, can you tell me how exactly we calculate the Error rate? The number of new errors over the selected time range? Not sure the label title and append copy are good enough...

Looks great otherwise 🙌

YulNaumenko

LGTM. Happy to see one more alerting integration!

formgeist

Some minor label changes that move for consistency across Observability apps.

formgeist · 2020-03-18T14:27:30Z

x-pack/legacy/plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/index.tsx

+                name: i18n.translate(
+                  'xpack.apm.serviceDetails.alertsMenu.viewActiveAlerts',
+                  {
+                    defaultMessage: 'View active alerts'


Suggested change

defaultMessage: 'View active alerts'

defaultMessage: 'Manage alerts'

formgeist · 2020-03-18T14:27:59Z

x-pack/legacy/plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/index.tsx

+const createThresholdAlertLabel = i18n.translate(
+  'xpack.apm.serviceDetails.alertsMenu.createThresholdAlert',
+  {
+    defaultMessage: 'Create threshold alert'


Suggested change

defaultMessage: 'Create threshold alert'

defaultMessage: 'Create alert'

formgeist · 2020-03-18T14:28:15Z

x-pack/legacy/plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/index.tsx

+          name: i18n.translate(
+            'xpack.apm.serviceDetails.alertsMenu.transactionDuration',
+            {
+              defaultMessage: 'Transaction duration'


Suggested change

defaultMessage: 'Transaction duration'

defaultMessage: 'Transaction duration threshold'

formgeist · 2020-03-18T14:28:25Z

x-pack/legacy/plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/index.tsx

+          name: i18n.translate(
+            'xpack.apm.serviceDetails.alertsMenu.errorRate',
+            {
+              defaultMessage: 'Error rate'


Suggested change

defaultMessage: 'Error rate'

defaultMessage: 'Error rate threshold'

kibanamachine · 2020-03-24T10:14:59Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: ecf61d6

History

💔 Build #35717 failed 5ff8d75
💔 Build #35653 failed bd7a6f01506447bfee7ff7adf753d56b821b2810
💚 Build #35314 succeeded cdf77cef1c76408c00362c85e7657a872cc6bb03
💚 Build #34667 succeeded fcb78c7e7bceb65c30c295bc53fa498537dedd03
💚 Build #33487 succeeded 5a286530fc12fdcf15a1921636991e6fb5b633b3

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Add alerting/actions permissions for APM * Export TIME_UNITS, getTimeUnitLabel from triggers actions UI plugin * Add APM alert types and UI * Review feedback * Use Expression components for triggers * Update alert name for transaction duration * Change defaults for error rate trigger

* master: Updating our direct usage of https-proxy-agent to 5.0.0 (elastic#58296) allow users to unset the throttle of an alert (elastic#60964) [Lens] Fix bug in metric config panel (elastic#60982) [SearchProfiler] Minor fixes (elastic#60919) [ML] Renaming ML setup and start contracts (elastic#60980) introduce StartServicesAccessor type for `CoreSetup.getStartServices` (elastic#60748) [SIEM][Detection Engine] Add rule's notification alert type (elastic#60832) [APM] Re-revert "Collect telemetry about data/API performance" (elastic#61030) [NP] Graph: get rid of saved objects class wrapper (elastic#59917) [EPM] merge duplicate fields when creating index patterns (elastic#60957) [Uptime] Ml detection of duration anomalies (elastic#59785) [Alerting] removes unimplemented buttons from Alert Details page (elastic#60934) [skip-ci] Fix CODEOWNERS paths for the Pulse team (elastic#60944) [APM] Threshold alerts (elastic#59566) [ML] Add support for percentiles aggregation to Transform wizard (elastic#60763) Cahgen save object duplicate message (elastic#60901)

cauemarcondes · 2020-04-09T11:56:20Z

Tests ok.

dgieselaar requested review from a team as code owners March 6, 2020 16:52

dgieselaar added release_note:enhancement v7.7.0 labels Mar 6, 2020

dgieselaar force-pushed the error-group-alert branch from a77f088 to 564dbc9 Compare March 9, 2020 12:38

sorenlouv reviewed Mar 10, 2020

View reviewed changes

.../plugins/apm/public/components/app/ServiceDetails/AlertIntegrations/AlertingFlyout/index.tsx Outdated Show resolved Hide resolved

sorenlouv reviewed Mar 12, 2020

View reviewed changes

dgieselaar force-pushed the error-group-alert branch from 564dbc9 to c0dc3c7 Compare March 12, 2020 13:11

sorenlouv approved these changes Mar 12, 2020

View reviewed changes

peterschretlen mentioned this pull request Mar 13, 2020

Add necessary privileges to apps using alerting #60135

Closed

3 tasks

YulNaumenko approved these changes Mar 13, 2020

View reviewed changes

dgieselaar force-pushed the error-group-alert branch from c0dc3c7 to 5a28653 Compare March 16, 2020 16:28

This was referenced Mar 17, 2020

[APM] Threshold alerts in APM elastic/apm#170

Closed

[APM] Display threshold alert annotations on time-series graphs #60373

Closed

formgeist reviewed Mar 18, 2020

View reviewed changes

dgieselaar mentioned this pull request Mar 19, 2020

[APM]: Replace error occurrence watchers with Kibana Alerting #46547

Closed

dgieselaar force-pushed the error-group-alert branch from 5a28653 to fcb78c7 Compare March 19, 2020 19:33

dgieselaar requested a review from a team as a code owner March 19, 2020 19:33

dgieselaar force-pushed the error-group-alert branch from fcb78c7 to cdf77ce Compare March 23, 2020 09:12

dgieselaar removed the request for review from a team March 23, 2020 19:27

dgieselaar force-pushed the error-group-alert branch 2 times, most recently from 4a9bcf5 to bd7a6f0 Compare March 23, 2020 22:20

dgieselaar added 6 commits March 24, 2020 08:31

Add alerting/actions permissions for APM

573570c

Export TIME_UNITS, getTimeUnitLabel from triggers actions UI plugin

6107771

Add APM alert types and UI

daea3ae

Review feedback

33acee5

Use Expression components for triggers

b326c42

Update alert name for transaction duration

5ff8d75

dgieselaar force-pushed the error-group-alert branch from bd7a6f0 to 5ff8d75 Compare March 24, 2020 08:24

Change defaults for error rate trigger

ecf61d6

dgieselaar merged commit 85c0be3 into elastic:master Mar 24, 2020

dgieselaar deleted the error-group-alert branch March 24, 2020 10:15

dgieselaar mentioned this pull request Mar 24, 2020

[7.x] [APM] Threshold alerts (#59566) #61046

Merged

cauemarcondes added the apm-test-plan-7.7.0 label Apr 6, 2020

cauemarcondes self-assigned this Apr 6, 2020

cauemarcondes added the apm:test-plan-done Pull request that was successfully tested during the test plan label Apr 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] Threshold alerts #59566

[APM] Threshold alerts #59566

dgieselaar commented Mar 6, 2020 •

edited

Loading

sorenlouv Mar 10, 2020

dgieselaar Mar 16, 2020

dgieselaar commented Mar 10, 2020 via email

sorenlouv Mar 12, 2020

sorenlouv Mar 12, 2020

sorenlouv Mar 12, 2020

sorenlouv Mar 12, 2020

dgieselaar Mar 16, 2020

sorenlouv Mar 12, 2020

sorenlouv Mar 12, 2020

dgieselaar Mar 16, 2020

sorenlouv Mar 16, 2020

sorenlouv Mar 12, 2020

sorenlouv Mar 12, 2020

dgieselaar Mar 16, 2020

sorenlouv Mar 16, 2020 •

edited

Loading

dgieselaar Mar 20, 2020

sorenlouv Mar 12, 2020

dgieselaar Mar 16, 2020

sorenlouv Mar 12, 2020

dgieselaar Mar 16, 2020

dgieselaar Mar 16, 2020

formgeist commented Mar 12, 2020

YulNaumenko left a comment

formgeist left a comment

formgeist Mar 18, 2020

formgeist Mar 18, 2020

formgeist Mar 18, 2020

formgeist Mar 18, 2020

kibanamachine commented Mar 24, 2020

cauemarcondes commented Apr 9, 2020

		`service.name:${params.serviceName}`,
		`transaction.type:${params.transactionType}`

	defaultMessage: 'View active alerts'
	defaultMessage: 'Manage alerts'

	defaultMessage: 'Create threshold alert'
	defaultMessage: 'Create alert'

	defaultMessage: 'Transaction duration'
	defaultMessage: 'Transaction duration threshold'

	defaultMessage: 'Error rate'
	defaultMessage: 'Error rate threshold'

[APM] Threshold alerts #59566

[APM] Threshold alerts #59566

Conversation

dgieselaar commented Mar 6, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgieselaar commented Mar 10, 2020 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv Mar 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

formgeist commented Mar 12, 2020

YulNaumenko left a comment

Choose a reason for hiding this comment

formgeist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Mar 24, 2020

💚 Build Succeeded

History

cauemarcondes commented Apr 9, 2020

dgieselaar commented Mar 6, 2020 •

edited

Loading

sorenlouv Mar 16, 2020 •

edited

Loading