[ResponseOps] UX for rules where interval is less than configured minimum

[ymao1]

The questions on the original issue have been answered so changing the issue description to reflect current scope, which is adding an indicator to the UI when a rule is below the configured minimum.

** Original Issue **
With this PR we have added a Kibana-wide configuration for the minimum schedule interval for a rule. This minimum will only apply to rules that are created or updated after this setting was added. Rules that were created before this setting was added will continue running.

There was a question raised about whether we should be able to enable/disable rules that were created before this setting was added and have a lower interval than the configuration

Questions: (updated with answers)

• Should users be able to disable and then re-enable a rule that was created before this setting and has a lower interval than the minimum?
  • Answer: Yes. Since this is the current behavior, no changes needed here.
• Related - when rules are imported, they are imported as disabled. Should users be able to enable imported rules that have a lower interval than the minimum?
  • Answer - Ideally, we want to update the interval for them on import if the minimum is lower than the schedule and enforce = true. This is blocked by [Saved Objects] Import hook/interceptor for saved object management #109189
• Should we show anything in the Rule Management UI when a rule has an interval lower than the minimum?
  • Answer - Yes

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[kobelb]

Should users be able to disable and then re-enable a rule that was created before this setting and has a lower interval than the minimum?

Ideally, yes. Does this add a lot of complexity?

Related - when rules are imported, they are imported as disabled. Should users be able to enable imported rules that have a lower interval than the minimum?

I don't think so, no. What if on on import we change the interval to be the minimum interval if it's lower than the minimum interval and display a warning?

Should we show anything in the Rule Management UI when a rule has an interval lower than the minimum?

Ideally, yes. Adding some type of indicator would be great.

[ymao1]

Should users be able to disable and then re-enable a rule that was created before this setting and has a lower interval than the minimum?

Ideally, yes. Does this add a lot of complexity?

No! That is the existing behavior. Just verifying that it is expected.

Related - when rules are imported, they are imported as disabled. Should users be able to enable imported rules that have a lower interval than the minimum?

I don't think so, no. What if on on import we change the interval to be the minimum interval if it's lower than the minimum interval and display a warning?

We can display a warning on import if there are rules below the configured minimum but are unable to modify rule SO on import until this issue is resolved.

[mikecote]

@XavierM regarding Should we show anything in the Rule Management UI when a rule has an interval lower than the minimum?. If you think we should, do you want us to open a separate issue?

[mikecote]

We also have this issue (#126946) which will cover the UX when the system warns users when they create rules with an interval below the configuration.

[ymao1]

@mdefazio We're looking to show an indicator on the Rule Management page when a rule's schedule is below the configured minimum. With this PR and this PR we're adding a configurable minimum interval that we'll treat as either a suggestion or a requirement. If it's a suggestion, we're not preventing users from creating these rules, just showing/logging some warnings.

Do you have any suggestions for what this indicator might look like? I was thinking it should be on the Interval column on the rule management table? Do you think it should look the same or different depending on whether we're strictly enforcing the minimum or not?

adding @ryankeairns too

[mdefazio]

@ymao1 Still thinking on this one a bit.

I think the interval column probably works for now, but wondering if we need to accommodate more things like this from a rule level --> warnings, insights, recommendations, etc. Maybe thinking ahead too much, but that's where I am.

When strictly enforcing it, would existing or imported rules fail because of this if they have intervals below the minimum? As opposed to allowing it and letting the rules run?

My thinking is that we make this warning the same regardless if suggesting or strict, and the rule would show differently anyway bc its failing if its strictly enforced.

[ymao1]

When strictly enforcing it, would existing or imported rules fail because of this if they have intervals below the minimum? As opposed to allowing it and letting the rules run?

@mdefazio The current thinking is that existing rules will continue to run even with strict enforcement. It will only be on update that we will enforce this minimum. For imported rules, we would like to eventually auto-adjust the interval on import to be at the minimum but this ability is not yet supported by core so for now they will continue to run.

[mdefazio]

After thinking on this further, and with other updates that will potentially effect the UI of the rule row/table, I'm thinking simply keeping this on the column for now until we can nail down an update for the full row and take this into account more holistically.

My reasoning is that this feels like a general rule warning and if there are future warnings we need to accommodate, they should be presented in a single metric, rather than scattered across columns. Perhaps there are opposing opinions to this, but my reaction is that improving how this is shown (outside of a simple alert icon in the column) involves reworking the whole row.

[ymao1]

@mdefazio That makes sense to me. Any suggestions for how an indicator on the column should look?

[mdefazio]

I was imagining a simple icon + tooltip after the interval value in the row. I was going between using a banner as well to let the user know there are rules with warnings. But I felt like this would be too obtrusive for this case. So I'm showing a toast instead. That said, are we able to filter these rules based on this kind of warning?

Prototype link

[ymao1]

@mdefazio We would not be able to filter these rules based on the warning. It would be nice to be able to sort on the interval column though, so users could see all their smaller interval rules grouped together. cc @XavierM

[mdefazio]

It would be nice to be able to sort on the interval column though, so users could see all their smaller interval rules grouped together

Would that actually solve the issue though? Wouldn't I be able to set a minimum interval of 1 day? Whereas other rules could have a minimum of 1 hour or 1 min?

Wouldn't this fall under a 'Warning' status (from the new proposed rule execution statuses)? So the inability to filter would only be until these new statuses are updated?

[ymao1]

The minimum is a global minimum and not per rule type so any rule that falls below the minimum would show this indicator on the rules list.

We are currently not putting the rule status into a warning state when the rule is below the configured minimum. That's an interesting idea, as I associated the warning status with something funky happened during rule execution and this seems more related to the management and configuration of the rules, where rules created with a schedule lower than the minimum will still run and executed successfully.

[mdefazio]

Ah, gotcha. But I guess then the issue is to let them know that I should sort by that column to find those rules with low intervals. Explaining that in the toast seems like it would get lost pretty quick. 🤔

Regarding your second point, I'm not 100% clear on the difference. A few reasons (which may or not be accurate though):

• This view is currently doing double duty on managing rules and monitoring rules at a high level. But once we have a specific view execution logs, wouldn't this type of warning still make sense in the management view? That said, if I am viewing a rule in a management view for instance, and don't see a warning, wouldn't it be confusing to see a warning when I see that rule in a log view context? And vice versa?
• Would this interval warning be visible on the rule detail view? Or only when I edit the rule and see the warning by the interval input? I can imagine it would be useful to show it at the detail level as well before having to edit the rule.
• While I understand why we divide them on our end. To the user, would it be clear? Would I need to look in two different places depending on the warning type?
• Couldn't this cause funky execution issues? Perhaps I'm not understanding the potential outcome.

[ymao1]

@mdefazio The goal of this minimum interval configuration is to encourage users to only create rules that are above the minimum because running a lot of rules with a small interval may affect alerting performance. That's not to say it necessarily does. It really depends on the number of rules they're running and whether they are all running at small intervals or whether they run a long time. If they're only running 1 rule at a 1 second, it's going to run just fine, even if it's below the minimum.

The execution statuses we have (error, warning) represent the state of the rule during the last execution. Right now, we only have 1 usage of warning which is if a rule scheduled a lot of actions, we're capping the maximum number that we're executing. So the rule executed successfully but the user might not get all of their notifications if the rule spawned > 100,000 actions.

To me there's a slight difference between (1) Warning because something actually happened during rule execution and (2) Warning because there may be a configuration issue that may or may not affect actual execution. Maybe I'm just too involved in the implementation details though and there's not actually much difference between the two as I think?

Would this interval warning be visible on the rule detail view? Or only when I edit the rule

Right now you would only see it when you edit the rule but I think it would makes sense to show something on the rule details view as well. I don't think we currently show the schedule interval on that view so we'd have to add it somewhere.

[mdefazio]

@ymao1 and I talked this through a bit more and reached the following consensus:

• We shouldn't show this as a warning icon to help differentiate from execution warnings. This is more of a recommendation that we hope will help us put stricter validation on this setting in the future.
• We are not able to filter down to these rules with this release, and so showing a toast without an action seems like it would be more frustrating than helpful—especially if the user is aware of this minimum and chooses to ignore it.
• It will be up to the user to find these rules—not ideal for large cardinality scenarios, but would be only for this release until we can provide more guidance / quick filters or sorting
• The icon (changing to an info icon) on the interval cell will trigger the edit rule flyout
• We will move the toast to the rule detail so it can provide an action to edit the rule settings
• Again, maybe not the most ideal experience right now, but we will work on improving how to get to these rules.
  Some ideas:
  • Perhaps an icon on the interval column heading
  • Maybe a tour component on the interval heading that we can then allow the user to dismiss and not show on every refresh.
  • Provide sorting on this column
  • Provide a space in the layout for 'improving your alert performance' that can house more of these types of scenarios.

I will update the mockups in this issue shortly.

[mdefazio]

Updated screenflow: