Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error message for partitioning by nested field in Iceberg #15142

Merged
merged 1 commit into from
Nov 30, 2022

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Nov 22, 2022

Description

Fixes #15091

@ebyhr ebyhr added the no-release-notes This pull request does not require release notes entry label Nov 22, 2022
@cla-bot cla-bot bot added the cla-signed label Nov 22, 2022
@ebyhr
Copy link
Member Author

ebyhr commented Nov 22, 2022

CI hit #14441

Copy link
Member

@alexjo2144 alexjo2144 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't mind adding some Spark compatibility test cases, here or in a separate PR:

  • Create a table in Spark partitioned on a nested field
    • Trino should be able to read it
    • Trino should not allow insert/update/delete/merge operations on it

@ebyhr ebyhr force-pushed the ebi/iceberg-nested-error-message branch from f30f7b7 to 34282e2 Compare November 24, 2022 03:19
@ebyhr ebyhr changed the title Improve error message when partitioning by nested fields in Iceberg Support reading partitioned tables by nested field and improve error message for writing operations in Iceberg Nov 24, 2022
@ebyhr ebyhr force-pushed the ebi/iceberg-nested-error-message branch 2 times, most recently from 44a4455 to 7976e06 Compare November 24, 2022 07:48
@ebyhr
Copy link
Member Author

ebyhr commented Nov 24, 2022

@alexjo2144 It seems the connector couldn't read such tables. I added another commit to fix this issue and updated more code to handle other statements. Could you take another look when you have time?

@ebyhr ebyhr force-pushed the ebi/iceberg-nested-error-message branch 2 times, most recently from 6f5806d to 8eb0b95 Compare November 25, 2022 07:33
Copy link
Member

@alexjo2144 alexjo2144 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we wait for a comment on if this should be supported from the Iceberg folks before merging the first commit? #15109 (comment)

@ebyhr ebyhr force-pushed the ebi/iceberg-nested-error-message branch from 8eb0b95 to 771a9ee Compare November 29, 2022 02:50
@ebyhr ebyhr changed the title Support reading partitioned tables by nested field and improve error message for writing operations in Iceberg Improve error message for partitioning by nested field in Iceberg Nov 29, 2022
@ebyhr ebyhr force-pushed the ebi/iceberg-nested-error-message branch from 771a9ee to ee4f31e Compare November 30, 2022 00:57
@ebyhr ebyhr merged commit 1e3e7c7 into trinodb:master Nov 30, 2022
@ebyhr ebyhr deleted the ebi/iceberg-nested-error-message branch November 30, 2022 04:28
@github-actions github-actions bot added this to the 404 milestone Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed no-release-notes This pull request does not require release notes entry
Development

Successfully merging this pull request may close these issues.

Improve error message when trying to partition on an Iceberg nested field
5 participants