Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Select from information schema leads to errors even when table filtered out #24448

Open
nikita-sheremet-java-developer opened this issue Dec 11, 2024 · 0 comments

Comments

@nikita-sheremet-java-developer

Consider select statement:

select * from yc_hive.information_schema.columns 
   where table_name != 'mytable'
   and (column_name = 'c1'
         or column_name = 'c2'
         or column_name = 'c3'
         or column_name = 'c4')
         
LIMIT 200

Expected result - all tables except mytable
Actual result:

io.trino.spi.TrinoException: Error listing table columns for catalog yc_hive: Metadata not found in metadata location for table test.mytable
	at io.trino.metadata.MetadataListing.handleListingException(MetadataListing.java:358)
	at io.trino.metadata.MetadataListing.listTableColumns(MetadataListing.java:254)
	at io.trino.connector.informationschema.InformationSchemaPageSource.addColumnsRecords(InformationSchemaPageSource.java:250)
	at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:218)
	at io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:185)
	at io.trino.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:381)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
	at io.trino.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
	at io.trino.operator.WorkProcessorUtils$BlockingProcess.process(WorkProcessorUtils.java:207)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:317)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:359)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
	at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
	at io.trino.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:255)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
	at io.trino.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:133)
	at io.trino.operator.Driver.processInternal(Driver.java:403)
	at io.trino.operator.Driver.lambda$process$8(Driver.java:306)
	at io.trino.operator.Driver.tryWithLock(Driver.java:709)
	at io.trino.operator.Driver.process(Driver.java:298)
	at io.trino.operator.Driver.processForDuration(Driver.java:269)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:890)
	at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
	at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
	at io.trino.$gen.Trino_464____20241209_170942_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: io.trino.spi.TrinoException: Metadata not found in metadata location for table test.mytable
	at io.trino.plugin.iceberg.CorruptedIcebergTableHandle.createException(CorruptedIcebergTableHandle.java:34)
	at io.trino.plugin.iceberg.IcebergMetadata.checkValidTableHandle(IcebergMetadata.java:3605)
	at io.trino.plugin.iceberg.IcebergMetadata.getTableMetadata(IcebergMetadata.java:735)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getTableMetadata(ClassLoaderSafeConnectorMetadata.java:280)
	at io.trino.tracing.TracingConnectorMetadata.getTableMetadata(TracingConnectorMetadata.java:250)
	at io.trino.metadata.MetadataManager.getTableMetadata(MetadataManager.java:473)
	at io.trino.tracing.TracingMetadata.getTableMetadata(TracingMetadata.java:302)
	at io.trino.metadata.MetadataListing.lambda$doListTableColumns$14(MetadataListing.java:328)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
	at io.trino.metadata.MetadataListing.doListTableColumns(MetadataListing.java:301)
	at io.trino.metadata.MetadataListing.listTableColumns(MetadataListing.java:251)
	... 41 more
Caused by: io.trino.spi.TrinoException: Metadata not found in metadata location for table test.mytable
	at io.trino.plugin.iceberg.IcebergExceptions.translateMetadataException(IcebergExceptions.java:48)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refreshFromMetadataLocation(AbstractIcebergTableOperations.java:272)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refreshFromMetadataLocation(AbstractIcebergTableOperations.java:239)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refresh(AbstractIcebergTableOperations.java:140)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.current(AbstractIcebergTableOperations.java:123)
	at io.trino.plugin.iceberg.catalog.hms.TrinoHiveCatalog.lambda$loadTable$11(TrinoHiveCatalog.java:448)
	at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4903)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
	at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4898)
	at io.trino.cache.EvictableCache.get(EvictableCache.java:118)
	at io.trino.cache.CacheUtils.uncheckedCacheGet(CacheUtils.java:39)
	at io.trino.plugin.iceberg.catalog.hms.TrinoHiveCatalog.loadTable(TrinoHiveCatalog.java:445)
	at io.trino.plugin.iceberg.IcebergMetadata.getTableHandle(IcebergMetadata.java:474)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getTableHandle(ClassLoaderSafeConnectorMetadata.java:1237)
	at io.trino.tracing.TracingConnectorMetadata.getTableHandle(TracingConnectorMetadata.java:142)
	at io.trino.metadata.MetadataManager.lambda$getTableHandle$5(MetadataManager.java:292)
	at java.base/java.util.Optional.flatMap(Optional.java:289)
	at io.trino.metadata.MetadataManager.getTableHandle(MetadataManager.java:283)
	at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1971)
	at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1960)
	at io.trino.tracing.TracingMetadata.getRedirectionAwareTableHandle(TracingMetadata.java:1494)
	at io.trino.metadata.MetadataListing.lambda$doListTableColumns$14(MetadataListing.java:309)
	... 52 more
Caused by: org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream for file: s3a://mybucket/mytable/metadata/00002-7784047f-6288-48eb-b436-ee079d9a354d.metadata.json
	at io.trino.plugin.iceberg.fileio.ForwardingInputFile.newStream(ForwardingInputFile.java:55)
	at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:279)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.lambda$refreshFromMetadataLocation$1(AbstractIcebergTableOperations.java:241)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.lambda$refreshFromMetadataLocation$3(AbstractIcebergTableOperations.java:266)
	at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
	at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
	at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
	at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
	at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
	at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refreshFromMetadataLocation(AbstractIcebergTableOperations.java:266)
	... 76 more
Caused by: java.io.FileNotFoundException: s3a://mybucket/mytable/metadata/00002-7784047f-6288-48eb-b436-ee079d9a354d.metadata.json
	at io.trino.filesystem.memory.MemoryFileSystemCache.handleException(MemoryFileSystemCache.java:192)
	at io.trino.filesystem.memory.MemoryFileSystemCache.getOrLoadFromCache(MemoryFileSystemCache.java:172)
	at io.trino.filesystem.memory.MemoryFileSystemCache.cacheStream(MemoryFileSystemCache.java:92)
	at io.trino.filesystem.tracing.TracingFileSystemCache.lambda$cacheStream$1(TracingFileSystemCache.java:66)
	at io.trino.filesystem.tracing.Tracing.withTracing(Tracing.java:47)
	at io.trino.filesystem.tracing.TracingFileSystemCache.cacheStream(TracingFileSystemCache.java:66)
	at io.trino.filesystem.cache.CacheInputFile.newStream(CacheInputFile.java:63)
	at io.trino.plugin.iceberg.fileio.ForwardingInputFile.newStream(ForwardingInputFile.java:52)
	... 86 more
Caused by: java.io.FileNotFoundException: s3a://mybucket/myrtable/metadata/00002-7784047f-6288-48eb-b436-ee079d9a354d.metadata.json
	at io.trino.filesystem.s3.S3InputFile.length(S3InputFile.java:79)
	at io.trino.filesystem.tracing.Tracing.withTracing(Tracing.java:47)
	at io.trino.filesystem.tracing.TracingInputFile.length(TracingInputFile.java:81)
	at io.trino.filesystem.memory.MemoryFileSystemCache.load(MemoryFileSystemCache.java:179)
	at io.trino.filesystem.memory.MemoryFileSystemCache.lambda$getOrLoadFromCache$3(MemoryFileSystemCache.java:169)
	at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4903)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
	at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4898)
	at io.trino.cache.EvictableCache.get(EvictableCache.java:118)
	at io.trino.filesystem.memory.MemoryFileSystemCache.getOrLoadFromCache(MemoryFileSystemCache.java:169)
	... 92 more

Yes metadata does not exist and this is ok for corrupted table that why it is filtered out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant