Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trino Hudi connector returning zero records for tables with partition projection enabled #24654

Open
drautela-scwx opened this issue Jan 8, 2025 · 0 comments

Comments

@drautela-scwx
Copy link

drautela-scwx commented Jan 8, 2025

Trino Hudi connector is returning zero records for tables with partition projection enabled. The connector works fine for tables that don't have partition projection enabled.

We are trying to setup the following in AWS:

Trino on EKS: version 465
HMS: Glue.

Queries for Hudi table with partition projection enabled is returning zero records with no errors.

Trino Hudi connector is working fine for Hudi tables that don't have partition projection enabled.

We have both Hive and Hudi connectors setup as follows:

(1) /etc/trino/catalog/awsdatacatalog.properties

connector.name=hive
hive.metastore=glue
hive.hive-views.enabled=true
hive.partition-projection-enabled=true
fs.native-s3.enabled=true
hive.hudi-catalog-name=hudi

(2) /etc/trino/catalog/hudi.properties

connector.name=hudi
hive.metastore=glue
fs.native-s3.enabled=true

Zero records are returned for Hudi tables whether we use awsdatacatalog or hudi catalog

select *
from awsdatacatalog.hudi_db.test_table_ro
where partition1 = "123"
and partition2 = "abc"
limit 10;
select *
from hudi.hudi_db.test_table_ro
where partition1 = "123"
and partition2 = "abc"
limit 10;
@drautela-scwx drautela-scwx changed the title Trino returning zero records for Hudi tables in AWS Glue Trino Hudi connector returning zero records for tables with partition projection enabled Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant