Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make aggregation statement compilation robust #2781

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Make aggregation statement compilation robust #2781

wants to merge 2 commits into from

Conversation

LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Jun 27, 2024 •
edited
Loading

Description

This PR is to address the weakness of query compilation for aggregation statement, including query with aggregate function, window function, group-by clause etc.

Here are some examples to demo what this PR fixes. The reproduce details is here

ID Query Description Before After (SemanticCheckException)
1 SELECT query_id FROM test_bulk1 GROUP BY name column not in group-by clause return null Field [query_id] must appear in the GROUP BY clause or be used in an aggregate function
2 SELECT query_id, max(query_id) FROM test_bulk1 group-by is missing Explicit GROUP BY clause is required because expression [query_id] contains non-aggregated column Explicit GROUP BY clause is required because expression [query_id] contains non-aggregated column
3 SELECT query_id FROM test_bulk1 GROUP BY max(query_id) aggregate functions in group-by Server side error during query execution Aggregate function is not allowed in a GROUP BY clause, but found [max]
4 SELECT max(query_id) FROM test_bulk1 GROUP BY 1 group-by position refers to aggregate function Server side error during query execution Aggregate function is not allowed in a GROUP BY clause, but found [max]
5 SELECT max(query_id) FILTER(WHERE query_id) FROM test_bulk1 non boolean filter in aggregate Client side error during query execution: [filter] must not be null FILTER or HAVING expression must be type boolean, but found [LONG]
6 SELECT max(query_id) FILTER(WHERE max(query_id)>10) FROM test_bulk1 aggregate in aggregate-filter error Server side error during query execution Aggregate function is not allowed in a FILTER, but found [max]
7 SELECT * FROM test_bulk1 WHERE ROW_NUMBER() OVER(ORDER BY name) > 0 window function in where-clause Server side error during query execution window functions are not allowed in WHERE or HAVING

Issues Resolved

Resolves #2767, #910,

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin
Make aggregation statement compilation robust
1c8a5ae 
Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin
Add user doc and integ tests
68bc359 
Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin LantaoJin marked this pull request as ready for review July 4, 2024 10:13
@LantaoJin
Copy link
Member Author

All checks passed on latest commit. @penghuo @dai-chen would you have a chance to review this PR?

@LantaoJin LantaoJin mentioned this pull request Sep 25, 2024
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ps48 ps48 Awaiting requested review from ps48 ps48 is a code owner

@kavithacm kavithacm Awaiting requested review from kavithacm kavithacm is a code owner

@derek-ho derek-ho Awaiting requested review from derek-ho derek-ho is a code owner

@joshuali925 joshuali925 Awaiting requested review from joshuali925 joshuali925 is a code owner

@dai-chen dai-chen Awaiting requested review from dai-chen dai-chen is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB is a code owner

@rupal-bq rupal-bq Awaiting requested review from rupal-bq rupal-bq is a code owner

@mengweieric mengweieric Awaiting requested review from mengweieric mengweieric is a code owner

@vamsi-amazon vamsi-amazon Awaiting requested review from vamsi-amazon vamsi-amazon is a code owner

@Swiddis Swiddis Awaiting requested review from Swiddis Swiddis is a code owner

@penghuo penghuo Awaiting requested review from penghuo penghuo is a code owner

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az is a code owner

@MaxKsyunz MaxKsyunz Awaiting requested review from MaxKsyunz MaxKsyunz is a code owner

@Yury-Fridlyand Yury-Fridlyand Awaiting requested review from Yury-Fridlyand Yury-Fridlyand is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha is a code owner

@forestmvey forestmvey Awaiting requested review from forestmvey forestmvey is a code owner

@acarbonetto acarbonetto Awaiting requested review from acarbonetto acarbonetto is a code owner

@GumpacG GumpacG Awaiting requested review from GumpacG GumpacG is a code owner

@ykmr1224 ykmr1224 Awaiting requested review from ykmr1224 ykmr1224 is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[RFC] Make aggregation statement compilation robust
1 participant
@LantaoJin