[alerts] provide an "explain" capability to show elasticsearch queries that alerts will run

[pmuellr]

Alerts are commonly implemented using elasticsearch queries built from the alert parameters provided by customers. While you can pretty much guess how those queries are built, it would be nice to provide some UX where a customer can show the exact query that will be used. Kind of like an "explain" for alerts.

This came up in the context of an SDH where a customer wanted to compare an elasticsearch query they wanted to use in an alert, to the alert parameters they thought would match. There's no way to do that comparison today, without looking at the code, and even that won't be easy since the query builders can get complex.

Example: provide a link in the alert parameter forms, which when clicked, will display a popup with the query used, filled in with parameters in the form.

Some notes/complications:

• alerts don't HAVE to be elasticsearch queries - my first thought was to expect alertTypes to return a JSON-able object which the UI would render nicely, but that won't work if your query was KQL/lucene syntax
• alerts could be making > 1 queries per alertType executor invocation, so we should allow for showing multiple queries, and no queries!
• how would we represent values used in the query that don't come from the params, but from the alert state or other dynamic sources that we won't have available; and if the alert type is doing multiple queries, the queries other than the first might use results from previous queries.
• this will be some new API that alertTypes will have to implement, probably make it optional to allow alertTypes to "opt-in" to this
• we'll need an HTTP endpoint the UI code can call to get the representation, and so should be in the alertsClient as well
• it would be nice to actually allow that query to be executed and show the results; maybe we don't need to go that far - if the query could run in the Dev Tools Console, we could allow a "clipboard paste" button, and navigation to the console, to allow it to be pasted in there and run ad hoc.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[mikecote]

@cnasikas was this issue re-added to the board for re-triage?

[cnasikas]

Hey @mikecote! I got pinged because of the Team:ResponseOps label and noticed it was on the old board but not the new one. This is why I added it again. Sorry if it was a mistake on my part.

[mikecote]

@cnasikas Gotcha, I see you added response-ops-mx-backlog which is the label we used on issues that are temporarily removed from the board. Perhaps we remove it again but keep the team label?

[cnasikas]

Oh, you are right! I forgot about it 😊. I removed it again.