Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid selecting unrelated schemas from the information_schema #11855

Merged

Conversation

findinpath
Copy link
Contributor

@findinpath findinpath commented Apr 7, 2022

Description

io.trino.plugin.iceberg.TestSharedHiveMetastore is based on AWS Glue. When trying to select all the table schemas on which is found a table named region or nation the routine may find a lot of unrelated schemas.

Moreover in the context of enabled table redirections, the tables retrieved for listing are inspected one by one in MetadataManager#listTables method. If one of the tables to be inspected, gets dropped, the test io.trino.plugin.iceberg.BaseSharedMetastoreTest#testReadInformationSchema fails.

Caused by: org.apache.iceberg.exceptions.RuntimeIOException: Failed to read file: HdfsInputFile{delegate=s3://***/iceberg_smoke_test_qkp9dzmdph/iceberg_smoke_test_qkp9dzmdph.db/nation/metadata/00000-862c01c1-eeb0-43bd-95f3-e1e7bd058336.metadata.json, identity=ConnectorIdentity{user='user', groups=[], principal=user, enabledSystemroles=[], extraCredentials=[]}}
	at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:253)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.lambda$refreshFromMetadataLocation$1(AbstractIcebergTableOperations.java:218)
	at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
	at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refreshFromMetadataLocation(AbstractIcebergTableOperations.java:217)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.refresh(AbstractIcebergTableOperations.java:122)
	at io.trino.plugin.iceberg.catalog.AbstractIcebergTableOperations.current(AbstractIcebergTableOperations.java:110)
	at io.trino.plugin.iceberg.catalog.glue.TrinoGlueCatalog.lambda$loadTable$5(TrinoGlueCatalog.java:274)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
	at io.trino.plugin.iceberg.catalog.glue.TrinoGlueCatalog.loadTable(TrinoGlueCatalog.java:264)
	at io.trino.plugin.iceberg.IcebergMetadata.getTableHandle(IcebergMetadata.java:217)
	at io.trino.plugin.iceberg.IcebergMetadata.getTableHandle(IcebergMetadata.java:160)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getTableHandle(ClassLoaderSafeConnectorMetadata.java:182)
	at io.trino.metadata.MetadataManager.lambda$getTableHandle$5(MetadataManager.java:279)
	at java.base/java.util.Optional.flatMap(Optional.java:294)
	at io.trino.metadata.MetadataManager.getTableHandle(MetadataManager.java:259)
	at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1484)
	at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1476)
	at io.trino.metadata.MetadataManager.isExistingRelationForListing(MetadataManager.java:544)
	at io.trino.metadata.MetadataManager.listTables(MetadataManager.java:510)
	at io.trino.metadata.MetadataListing.listTables(MetadataListing.java:112)
	at io.trino.connector.informationschema.InformationSchemaPageSource.addTablesRecords(InformationSchemaPageSource.java:281)
	at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:219)

This is not a fix for the issue, but rather a patch to avoid having the Glue test which rely on shared metastore infrastructure to fail.

Related issues

#9400

PR fixing this issue #11790

@findinpath findinpath force-pushed the fix-iceberg-glue-shared-metastore-tests branch from 5452098 to a286d56 Compare April 7, 2022 20:02
@findepi
Copy link
Member

findepi commented Apr 8, 2022

created #11868

@findepi findepi added test no-release-notes This pull request does not require release notes entry labels Apr 8, 2022
@findepi findepi merged commit da58cc0 into trinodb:master Apr 8, 2022
@github-actions github-actions bot added this to the 377 milestone Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed no-release-notes This pull request does not require release notes entry test
Development

Successfully merging this pull request may close these issues.

2 participants