-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ignore nested delta directories for Hive ACID #20840
Ignore nested delta directories for Hive ACID #20840
Conversation
...rino-product-tests/src/main/java/io/trino/tests/product/hive/TestHiveTransactionalTable.java
Show resolved
Hide resolved
8b21aba
to
e0271c8
Compare
...rino-product-tests/src/main/java/io/trino/tests/product/hive/TestHiveTransactionalTable.java
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/util/AcidTables.java
Show resolved
Hide resolved
e0271c8
to
c1ee65a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change seems good, but needs a test in TestHiveAcidUtils
.
This doesn't seem to fix #8920 as that is a different error message and comes from BackgroundHiveSplitLoader
. Do you want to fix that as well?
plugin/trino-hive/src/main/java/io/trino/plugin/hive/util/AcidTables.java
Show resolved
Hide resolved
plugin/trino-hive/src/test/java/io/trino/plugin/hive/util/TestAcidTables.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs a test in TestHiveAcidUtils
c1ee65a
to
b171307
Compare
Implementation follows Hive behaviour
b171307
to
e819407
Compare
Thank you for review @electrum
Added.
Error message and origin for this ACID error changed since issue was created. I guess it was changed with converting AcidUtils to Trino with 540f53e commit. |
Description
Fixes: #16315
Related: #8920
Multiple Hive minor compaction creates duplicated directoires like:
Hive is able to read such table without problem, ignoring nested delta directories. This change follows Hive behaviour.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: