feat(snomed.datastore): Adjust page size dynamically for streaming queries #1187
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
...requested items is larger than the currently configured maximum result window.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## 8.x #1187 +/- ##
=========================================
Coverage 64.01% 64.01%
- Complexity 12805 12810 +5
=========================================
Files 1764 1765 +1
Lines 59773 59836 +63
Branches 5528 5529 +1
=========================================
+ Hits 38264 38307 +43
- Misses 19132 19151 +19
- Partials 2377 2378 +1
☔ View full report in Codecov by Sentry. |
...first batch did not produce any results (eg. due to an impossible "searchAfter" value). In these cases remainingCount is equal to totalHitCount, but the "while" loop exits immediately as there is no "next searchAfter" value to extract.
...exceeds the result window limit
These are limited by both the maximum amount of allowed terms in a TermsQuery as well as the maximum result set size in "result_window".
...SnomedConceptUpdateRequest
cmark
requested changes
Jul 11, 2023
| final Query<Data> query = Query.select(Data.class) | ||
| .where(Expressions.matchAll()) | ||
| .limit(1000) | ||
| .limit(pageSize) |
There was a problem hiding this comment.
I think the purpose here was to actually page through the results with the page size of 1000, now we are going to fetch it in one go, right?
| @@ -194,7 +194,7 @@ public <T> Hits<T> search(Query<T> query) throws IOException { | |||
| final boolean isLiveStreaming = !Strings.isNullOrEmpty(query.getSearchAfter()); | |||
| if (isLocalStreaming) { | |||
| checkArgument(!isLiveStreaming, "Cannot use searchAfter when requesting more items (%s) than the configured result window (%s).", limit, resultWindow); | |||
| } else if (isLiveStreaming) { | |||
| } if (isLiveStreaming) { | |||
apeteri
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The limit should never exceed the value of
result_windowconfigured for Elasticsearch.