Painless: More Safety Features

[jdconrad]

Painless currently utilizes an improvised statement counter in an attempt to prevent infinite loops. In some use cases this isn't really enough as each iteration of the loop can take an excessive amount of time. We should explore the possibility of adding additional safety features.

Some ideas that have been proposed already are the following:

• add a sampling of nanoseconds where every n iterations of a loop, time is taken into account to possibly terminate a script early if it's taking too long
• track memory usage of certain types of objects including Strings, and have a cap for allowed memory usage
• possibly create some custom data structures to better track usage of memory and number of API calls

Given that one of Painless's primary design goals is performance, I would like to propose a final feature if we do decide we would like more safety features -- two modes, safety and performance. As each of the above described features would likely add a non-insignificant amount of overhead, we could have a mode where they are disabled if the user requires the best performance available. Or we could even have a selection of enabled/disabled safety features, but this may be overly complex.

I would be interested in hearing about everyone's ideas, thoughts, and concerns related to this issue. @simonw @nik9000 @rjernst

[elasticmachine]

Pinging @elastic/es-core-infra

[jdconrad]

For regex this might be a good solution (https://www.exratione.com/2017/06/preventing-unbounded-regular-expression-operations-in-java/) -- we could use a counter in place of system time to avoid system calls or something along those lines instead.

[nik9000]

I've thought about things like that on and off for a little while. Something like this could be useful in a few places in Elasticsearch where we use regexes.

…

On Thu, May 31, 2018 at 3:54 PM Jack Conradson ***@***.***> wrote: For regex this might be a good solution ( https://www.exratione.com/2017/06/preventing-unbounded-regular-expression-operations-in-java/) -- we could use a counter in place of system time to avoid system calls or something along those lines instead. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANLojKpbeDff9ZFgF87D2IhM-UzXm_7ks5t4EplgaJpZM4TjtcH> .

[henningandersen]

Using the scripted metric aggregation, it is possible to build up large maps that can OOM the node without any circuit breaking. It would be nice to have a way of circuit breaking during scripting when the data that the script holds on to grows too big (or at least give the parent circuit breaker a chance to trip).

[rjernst]

@henningandersen Do you mean specifically with the map script, or with combine and/or reduce? In any case, I think this can be done by having "smart" objects which are passed into the script instead of plain Maps. This way after the agg calls execute for each document, it can update the circuit breaker with any change in the data size.

[henningandersen]

@rjernst I think all 3 scripts could experience this. The map script is probably the primary way too much data is collected, but the combine (and reduce) script then risks gathering a similar amount (but in one script invocation).
A complexity in finding out the memory usage is shared objects, where combine in some cases returns the collected datastructure directly and in other cases a new structure, could be a map with same keys, but other values.

[stu-elastic]

Closed in favor of #49871 #49872 #49873