-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve Iceberg deletes when an entire file can be removed #14198
Improve Iceberg deletes when an entire file can be removed #14198
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we perhaps have a $files
-based test that shows that this actually works as expected?
7cb32ba
to
560f01e
Compare
@findepi just pushed a fixup to one of the
This is essentially what the tests in TestIcebergV2 are doing, they just hit the Iceberg API directly instead of querying $files. But I can add them if you want. |
plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergAnalyze.java
Show resolved
Hide resolved
👍 |
This reverts commit d435549.
560f01e
to
482013d
Compare
squashed |
@alexjo2144 do you remember what RNs we had for this? |
|
Description
This reverts commit d435549.
There were some minor conflicts due to the change to TrinoFileSystem
Replace the feature where if a whole file is deleted by a single operation, remove it from the Iceberg manifest rather than writing position deletes.
Non-technical explanation
Improve performance of large delete operations.
Related PRs
Original changes: #12197
Revert: #13485
Iceberg core issue: apache/iceberg#5442
Iceberg dep. version bump: #14074
Release notes
( ) This is not user-visible and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: