Fix reading Delta Table metadata from Glue #12164
Conversation
The location field can sometimes be null (e.g. Iceberg tables, and sometimes Delta Lake tables).
Delta Lake connector expects that the metastore storage descriptor be fully translated if it exists. It only needs to return a dummy storage value if no storage descriptor has been configured.
findepi
changed the title
|
Test PR: #12172 |
|
@martint @electrum @dain @alexjo2144 .. this fixes the current release blocker... if we get this merged @martint and myself can get the release out today potentially. |
| // Default getter requires location to be set. Location is not set in the following scenarios: | ||
| // 1. CREATE TABLE flow in Hive connector sets delegate-transactional-managed-table-location-to-metastore to true and location is determined by HMS. | ||
| // 2. Iceberg format tables | ||
| // 3. Delta format tables (sometimes) |
There was a problem hiding this comment.
I double checked this today. Looks like we always expect the path to be in the storage descriptor. We could change that for Managed tables, but right now it's required.
There was a problem hiding this comment.
@alexjo2144, do you want me to remove the assumption that storage won't be specified for delta tables then?
There was a problem hiding this comment.
I think this change is fine, I made an Issue for follow up #12180
|
CI #12151 |
|
@findepi, it looks like the CI build with secrets failed on something that is transient. Can you kick it again and see if it passes? |
|
Does anyone else have any concerns here? Otherwise, I think this is good to go. |
Cherry-pick of trinodb/trino#12164 Delta Lake connector expects that the metastore storage descriptor be fully translated if it exists. It only needs to return a dummy storage value if no storage descriptor has been configured. Co-authored-by: Eric Hwang <eric.hwang@starburstdata.com>
Cherry-pick of trinodb/trino#12164 Delta Lake connector expects that the metastore storage descriptor be fully translated if it exists. It only needs to return a dummy storage value if no storage descriptor has been configured. Co-authored-by: Eric Hwang <eric.hwang@starburstdata.com>
Fixes: #12147
Caused by this PR:
#12130