Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: tests for LOOKUP JOIN with non-unique join keys #118471

Merged

Conversation

alex-spies
Copy link
Contributor

@alex-spies alex-spies commented Dec 11, 2024

Add a csv dataset and tests for LOOKUP JOIN where the join keys are not unique. In particular, add tests that include MVs and nulls to see how LOOKUP JOIN treats these.

Comment on lines +3 to +4
1,English,
1,,United Kingdom
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some intentional denormalization ("dirty data") to show how we're collecting nulls and mvs.

@alex-spies alex-spies added >test Issues or PRs that are addressing/adding tests auto-backport Automatically create backport pull requests when merged :Analytics/ES|QL AKA ESQL v8.18.0 labels Dec 11, 2024
@alex-spies alex-spies marked this pull request as ready for review December 11, 2024 17:42
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Dec 11, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Contributor

@craigtaverner craigtaverner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I would prefer the tests got moved to before the clientips tests to reduce conflict with my PR.

@@ -3,7 +3,6 @@
// Reuses the sample dataset and commands from enrich.csv-spec
//

//TODO: this sometimes returns null instead of the looked up value (likely related to the execution order)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this in my PR too!

Just after the tests with the languages_lookup index, for better
organization of the tests.
@alex-spies alex-spies added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Dec 12, 2024
@elasticsearchmachine elasticsearchmachine merged commit ccdea4a into elastic:main Dec 13, 2024
16 checks passed
@alex-spies alex-spies deleted the lookup-join-non-unique-tests branch December 13, 2024 12:42
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

alex-spies added a commit to alex-spies/elasticsearch that referenced this pull request Dec 13, 2024
Add a csv dataset and tests for `LOOKUP JOIN` where the join keys are
not unique. In particular, add tests that include MVs and nulls to see
how `LOOKUP JOIN` treats these.
elasticsearchmachine pushed a commit that referenced this pull request Dec 16, 2024
)

Add a csv dataset and tests for `LOOKUP JOIN` where the join keys are
not unique. In particular, add tests that include MVs and nulls to see
how `LOOKUP JOIN` treats these.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test Issues or PRs that are addressing/adding tests v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants