Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run terms concurrently when cardinality is only lower than shard size #110369

Merged
merged 2 commits into from
Jul 2, 2024

Conversation

iverase
Copy link
Contributor

@iverase iverase commented Jul 2, 2024

We currently run terms aggregations concurrently when cardinality is lower or equal to shard size. The issue is that when merging internal aggregations for cardinalities equal to the shard size, we don't know if we have truncate the list of terms, so an error is computed which is not expected.

This is happening here where the condition is size < terms.getShardSize():

if (size == 0 || size < terms.getShardSize() || isKeyOrder(terms.getOrder())) {

To make sure the results are the same between concurrent and non-concurrent runs, we need to exclude situations where shard size == cardinality. This showed up in a random test.

fixes #105505

@iverase iverase requested a review from javanna July 2, 2024 10:00
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jul 2, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine
Copy link
Collaborator

Hi @iverase, I've created a changelog YAML for you.

@iverase iverase merged commit 2441340 into elastic:main Jul 2, 2024
15 checks passed
@iverase iverase deleted the TermsConcurrency2 branch July 2, 2024 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] TermsDocCountErrorIT testStringValueFieldSingleShard failing
3 participants