Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable caching deterministic scripted queries in shard request cache #49466

Closed
stu-elastic opened this issue Nov 21, 2019 · 2 comments
Closed

Enable caching deterministic scripted queries in shard request cache #49466

stu-elastic opened this issue Nov 21, 2019 · 2 comments
Assignees
@stu-elastic
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >enhancement v8.0.0-alpha1

Comments

@stu-elastic
Copy link
Contributor

Related: #49321

Right now, there is no way to know if scripts are deterministic or not. To ensure correctness, these scripts skip the shard request cache.

Painless scripts are deterministic if they only use deterministic APIs, eg not randomUUID. We'll have to evaluate other languages.

If a script is deterministic, it's results are cachable.
@stu-elastic stu-elastic added >enhancement :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache v8.0.0 labels Nov 21, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Scripting)

@stu-elastic stu-elastic self-assigned this Nov 21, 2019
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Dec 6, 2019
@stu-elastic
Scripting: Groundwork for caching script results
1eb7045 
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: elastic#49466
@stu-elastic stu-elastic mentioned this issue Dec 6, 2019
Merged
stu-elastic added a commit that referenced this issue Dec 6, 2019
@stu-elastic
Scripting: Groundwork for caching script results (#49895)
356d1a2 
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: #49466
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Dec 6, 2019
@stu-elastic
Scripting: Groundwork for caching script results (elastic#49895)
81ec22a 
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: elastic#49466
@stu-elastic stu-elastic mentioned this issue Dec 6, 2019
Merged
stu-elastic added a commit that referenced this issue Dec 6, 2019
@stu-elastic
Scripting: Groundwork for caching script results (#49895) (#49944)
17cda5b 
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: #49466

**Backport**
@stu-elastic stu-elastic mentioned this issue Dec 11, 2019
Merged
stu-elastic added a commit that referenced this issue Dec 18, 2019
@stu-elastic
Scripting: Cache script results if deterministic (#50106)
7794625 
Cache results from queries that use scripts if they use only
deterministic API calls.  Nondeterministic API calls are marked in the
whitelist with the `@nondeterministic` annotation.  Examples are
`Math.random()` and `new Date()`.

Refs: #49466
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Dec 18, 2019
@stu-elastic
Scripting: Cache script results if deterministic (elastic#50106)
e02e10c 
Cache results from queries that use scripts if they use only
deterministic API calls.  Nondeterministic API calls are marked in the
whitelist with the `@nondeterministic` annotation.  Examples are
`Math.random()` and `new Date()`.

Refs: elastic#49466
@stu-elastic stu-elastic mentioned this issue Dec 18, 2019
Merged
stu-elastic added a commit that referenced this issue Dec 18, 2019
@stu-elastic
Scripting: Cache script results if deterministic (#50106) (#50329)
06a24f0 
Cache results from queries that use scripts if they use only
deterministic API calls.  Nondeterministic API calls are marked in the
whitelist with the `@nondeterministic` annotation.  Examples are
`Math.random()` and `new Date()`.

Refs: #49466
@stu-elastic stu-elastic mentioned this issue Dec 18, 2019
Closed
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Dec 18, 2019
@stu-elastic
Scripting: ScriptFactory not required by compile
79d1263 
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

Relates: elastic#49466
This was referenced Dec 18, 2019
Merged
Open
stu-elastic added a commit that referenced this issue Dec 19, 2019
@stu-elastic
Scripting: ScriptFactory not required by compile (#50344)
cd721b6 
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

Relates: #49466
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Dec 19, 2019
@stu-elastic
Scripting: ScriptFactory not required by compile (elastic#50344)
4249af9 
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

Relates: elastic#49466
@stu-elastic stu-elastic mentioned this issue Dec 19, 2019
Merged
stu-elastic added a commit that referenced this issue Dec 19, 2019
@stu-elastic
Scripting: ScriptFactory not required by compile (#50344) (#50392)
689df1f 
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

**Backport**

Relates: #49466
@stu-elastic
Copy link
Contributor Author

Completed and available in 8.0 and 7.6

SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this issue Jan 23, 2020
@stu-elastic @SivagurunathanV
Scripting: Groundwork for caching script results (elastic#49895)
f2b233c 
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: elastic#49466
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this issue Jan 23, 2020
@stu-elastic @SivagurunathanV
Scripting: Cache script results if deterministic (elastic#50106)
85d6941 
Cache results from queries that use scripts if they use only
deterministic API calls.  Nondeterministic API calls are marked in the
whitelist with the `@nondeterministic` annotation.  Examples are
`Math.random()` and `new Date()`.

Refs: elastic#49466
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this issue Jan 23, 2020
@stu-elastic @SivagurunathanV
Scripting: ScriptFactory not required by compile (elastic#50344)
a54d6cb 
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

Relates: elastic#49466
This was referenced Feb 3, 2020
Closed
Closed
@spiritinlife spiritinlife mentioned this issue Apr 25, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stu-elastic stu-elastic

Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >enhancement v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jakelandis @elasticmachine @stu-elastic