EQL: Introduce sequence internal paging #58859
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refactor sequence matching classes in order to decouple querying from results consumption (and matching). Rename some classes to better convey their intent. Introduce internal pagination of sequence algorithm, that is getting the data in slices and, if needed, moving forward in order to find more matches until either the dataset is consumer or the number of results desired is found.
|
Pinging @elastic/es-ql (:Query Languages/EQL) |
costin
commented
Jul 1, 2020
| /** | ||
| * Executable tracking sequences at runtime. | ||
| */ | ||
| class Matcher { |
There was a problem hiding this comment.
This used to be inside SequenceRuntime but now it has been extracted and takes care only of matching.
costin
commented
Jul 1, 2020
| * This allows the window to find any follow-up results even if they are found outside the initial window | ||
| * of a base query. | ||
| */ | ||
| public class TumblingWindow implements Executable { |
There was a problem hiding this comment.
This is the querying/pagination part from SequenceRuntime; in this version it introduces two new things:
- range queries to the current window (instead of being open ended which made it hard to reason on whether a sequence should be matched or data be discarded)
- pagination/advancement. If there's no more data in the current query, keep looking in the other queries (as long as they have something to match against).
astefan
approved these changes
Jul 2, 2020
| String timestampName = Expressions.name(timestamp); | ||
| String tiebreakerName = Expressions.isPresent(tiebreaker) ? Expressions.name(tiebreaker) : null; | ||
|
|
||
| Criterion<QueryRequest> base = null; |
There was a problem hiding this comment.
This one doesn't seem to actually be used.
|
|
||
| BoxedQueryRequest boxedRequest = new BoxedQueryRequest(original, timestampName, tiebreakerName); | ||
| Criterion<BoxedQueryRequest> criterion = | ||
| new Criterion<>(i, boxedRequest, keyExtractors, tsExtractor, tbExtractor, i> 0 && descending); |
There was a problem hiding this comment.
descending can be false so i > 0 && descending != i > 0
There was a problem hiding this comment.
I think it was about the formatting (missing space)
costin
added a commit
that referenced
this pull request
Jul 2, 2020
Refactor sequence matching classes in order to decouple querying from results consumption (and matching). Rename some classes to better convey their intent. Introduce internal pagination of sequence algorithm, that is getting the data in slices and, if needed, moving forward in order to find more matches until either the dataset is consumer or the number of results desired is found. (cherry picked from commit bcf2c11)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactor sequence matching classes in order to decouple querying from
results consumption (and matching).
Rename some classes to better convey their intent.
Introduce internal pagination of sequence algorithm, that is getting the
data in slices and, if needed, moving forward in order to find more
matches until either the dataset is consumer or the number of results
desired is found.