Allow moving_function to access percentile sketches #49452
Labels
:Analytics/Aggregations
Aggregations
>enhancement
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
If the
moving_function
pipeline agg could access and merge the internal percentiles sketch (HDRHisto or TDigest), it would allow users to construct moving windows of percentiles. E.g. bucket the results at 1 minute intervals and apply apercentiles
agg per-bucket, then using amoving_function
over 5 buckets and inside the script merge the sketches to produce a single sketch for that window of time.When plotted, this gives a percentiles output of the last five minutes, every minute.
This is similar to what the cumulative cardinality agg does, except merging windows of percentiles together instead of cumulatively merging a cardinality sketch.
I think the main difficulty will be exposing this functionality in a safe/simple manner to painless. I think we would first need to allow percentiles objects as a valid input (instead of just numerics/multi-valued stats) and then build a dedicated painless function similar to the existing moving averages (e.g.
MovingFunctions.percentiles(values)
)If we can't find a good way to do this, we could always create a specialized "moving percentiles" pipeline agg but it would probably be better to extend
moving_function
first if possible.The text was updated successfully, but these errors were encountered: