Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Glue metastore in Iceberg connector #9363

Closed
LuPan2015 opened this issue Sep 24, 2021 · 3 comments
Closed

Support Glue metastore in Iceberg connector #9363

LuPan2015 opened this issue Sep 24, 2021 · 3 comments
Labels
enhancement New feature or request

Comments

@LuPan2015
Copy link

LuPan2015 commented Sep 24, 2021

  1. version: trino 359
  2. iceberg.properties
connector.name=iceberg
hive.metastore = glue
  1. execute command
show tables
  1. I get the following exception
2021-09-24T01:44:55.632Z        ERROR   SplitRunner-2-112       io.trino.execution.executor.TaskExecutor        Error processing Split 20210924_014455_00006_yff5w.1.0-0 io.trino.connector.informationschema.InformationSchemaSplit@29d90dd3 (start = 1390712.491091, wall = 9 ms, cpu = 0 ms, wait = 0 ms, calls = 1)
java.lang.UnsupportedOperationException: getTablesWithParameter for GlueHiveMetastore is not implemented
        at io.trino.plugin.hive.metastore.glue.GlueHiveMetastore.getTablesWithParameter(GlueHiveMetastore.java:448)
        at io.trino.plugin.iceberg.IcebergMetadata.lambda$listTables$9(IcebergMetadata.java:403)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
        at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747)
        at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at io.trino.plugin.iceberg.IcebergMetadata.listTables(IcebergMetadata.java:408)
        at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.listTables(ClassLoaderSafeConnectorMetadata.java:263)
        at io.trino.metadata.MetadataManager.listTables(MetadataManager.java:631)
        at io.trino.metadata.MetadataListing.listTables(MetadataListing.java:98)
        at io.trino.connector.informationschema.InformationSchemaPageSource.addTablesRecords(InformationSchemaPageSource.java:281)
        at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:219)
        at io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:183)
        at io.trino.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:379)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
        at io.trino.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
        at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:277)
        at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:319)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
        at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
        at io.trino.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:215)
        at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
        at io.trino.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:150)
        at io.trino.operator.Driver.processInternal(Driver.java:387)
        at io.trino.operator.Driver.lambda$processFor$9(Driver.java:291)
        at io.trino.operator.Driver.tryWithLock(Driver.java:683)
        at io.trino.operator.Driver.processFor(Driver.java:284)
        at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
        at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
        at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
        at io.trino.$gen.Trino_359____20210924_014338_2.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
@findepi findepi changed the title iceberg connector hive.metastore is glue not support Support Glue metastore in Iceberg connector Sep 24, 2021
@findepi findepi added the enhancement New feature or request label Sep 24, 2021
@findepi findepi mentioned this issue Sep 24, 2021
93 tasks
@mailtoraja18
Copy link

looking forward for this enhancement. any eta ?

@wrb2
Copy link

wrb2 commented Jan 27, 2022

Having this (or the dynamo catalog, I guess) would enable us using Iceberg.

@findepi
Copy link
Member

findepi commented Mar 15, 2022

Addressed by #10845

@findepi findepi added this to the 374 milestone Mar 15, 2022
@martint martint removed this from the 374 milestone Mar 16, 2022
@ebyhr ebyhr closed this as completed Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

6 participants