Percolate Query implementation in Document Level Alerting

[sbcd90]

Is your feature request related to a problem? Please describe.

OpenSearch Alerting plugin allows users to monitor different data sources (e.g. indices) & subsequently create alerts when the trigger condition is met. One of the important features that is being worked upon now is Document Level Alerting.

Document Level Alerting can be briefly defined in 2 steps. In the first step, documents created within a fixed time interval in a customer created index are matched against a set of queries. Now, in the second step, these document-query pairs are matched against a trigger condition & if it is match, an alert is triggered.

Hence, this feature fits more into the percolate query category as the queries can be stored into an index with a percolator field & documents can then be matched against it.

Describe the solution you'd like

The following approach looked the most feasible & was chosen for implementation.

In this approach, as shown in the above diagram, there is only 1 central percolate index .opendistro-alerting-queries.

Every time the alerting plugin receives a create monitor or update monitor call, it first pulls the updated latest mapping of customer index & then runs a diff check on whether there is any difference in mapping between customer index & central percolator index.

Based on the diff checker results, fields are dynamically updated in the mapping of central percolator index & then queries are inserted.

[engechas]

Closing this as completed