Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unintended HiveConf overriding in Coral leading to query execution auth failures #534

Closed
1fanwang opened this issue Sep 20, 2024 · 2 comments · Fixed by #539
Closed

Unintended HiveConf overriding in Coral leading to query execution auth failures #534

1fanwang opened this issue Sep 20, 2024 · 2 comments · Fixed by #539
Labels
help wanted Extra attention is needed

Comments

@1fanwang
Copy link
Collaborator

1fanwang commented Sep 20, 2024 •
edited
Loading

Problem Description

Coral is experiencing authentication failures when accessing Hive metastore data due to unintended hijacking/overriding of HiveConf objects across different components. Introduced from #503

**Example auth failure logs <- expand here** 
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 24/09/16 14:34:49 WARN annotation.RetryTranslator: Exception in retry: Query failed (#20240916_143448_00020_gp9sb): Failed to translate Hive view 'coral_schema.coral_table_view': Unable to determine current user. Please refer to go/coral/unsupported (help: go/dalisupportal) class: java.sql.SQLException
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 24/09/16 14:34:49 WARN annotation.RetryTranslator: Exception in retry: Failed to translate Hive view 'coral_schema.coral_table_view': Unable to determine current user. Please refer to go/coral/unsupported (help: go/dalisupportal) class: io.prestosql.jdbc.$internal.client.FailureInfo$FailureException
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 24/09/16 14:34:49 WARN annotation.RetryTranslator: Exception in retry: Unable to determine current user class: io.prestosql.jdbc.$internal.client.FailureInfo$FailureException
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 24/09/16 14:34:49 WARN annotation.RetryTranslator: Exception in retry: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:862)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:824)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:693)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:252)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:220)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:212)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:909)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5147)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5099)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.parsetree.parser.CoralParseDriver.parse(CoralParseDriver.java:45)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.parsetree.ParseTreeBuilder.process(ParseTreeBuilder.java:123)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.HiveToRelConverter.toSqlNode(HiveToRelConverter.java:100)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.common.ToRelConverter.processView(ToRelConverter.java:152)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.common.ToRelConverter.convertView(ToRelConverter.java:123)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.ViewReaderUtil$HiveViewReader.decodeViewData(ViewReaderUtil.java:218)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.lambda$toConnectorViewDefinition$69(HiveMetadata.java:2600)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.Optional.map(Optional.java:260)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.toConnectorViewDefinition(HiveMetadata.java:2594)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.getView(HiveMetadata.java:2587)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getView(ClassLoaderSafeConnectorMetadata.java:613)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.metadata.MetadataManager.getViewInternal(MetadataManager.java:1206)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.metadata.MetadataManager.getView(MetadataManager.java:1181)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:2161)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.Table.accept(Table.java:60)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeFrom(StatementAnalyzer.java:4452)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:2914)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:508)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1468)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.Query.accept(Query.java:107)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:462)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:79)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:71)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:258)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:196)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:818)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:143)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.$gen.Trino_406_171_17____20240916_140731_2.call(Unknown Source)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.lang.Thread.run(Thread.java:840)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO -  class: io.prestosql.jdbc.$internal.client.FailureInfo$FailureException
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 24/09/16 14:34:49 WARN annotation.RetryTranslator: Exception in retry: java.lang.NullPointerException: invalid null input: name
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:862)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:824)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:693)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:252)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:220)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:212)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:909)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5147)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5099)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.parsetree.parser.CoralParseDriver.parse(CoralParseDriver.java:45)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.parsetree.ParseTreeBuilder.process(ParseTreeBuilder.java:123)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.hive.hive2rel.HiveToRelConverter.toSqlNode(HiveToRelConverter.java:100)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.common.ToRelConverter.processView(ToRelConverter.java:152)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.linkedin.coral.common.ToRelConverter.convertView(ToRelConverter.java:123)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.ViewReaderUtil$HiveViewReader.decodeViewData(ViewReaderUtil.java:218)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.lambda$toConnectorViewDefinition$69(HiveMetadata.java:2600)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.Optional.map(Optional.java:260)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.toConnectorViewDefinition(HiveMetadata.java:2594)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.hive.HiveMetadata.getView(HiveMetadata.java:2587)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getView(ClassLoaderSafeConnectorMetadata.java:613)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.metadata.MetadataManager.getViewInternal(MetadataManager.java:1206)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.metadata.MetadataManager.getView(MetadataManager.java:1181)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:2161)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.Table.accept(Table.java:60)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeFrom(StatementAnalyzer.java:4452)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:2914)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:508)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1468)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:483)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.Query.accept(Query.java:107)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:500)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:462)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:79)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:71)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:258)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:196)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:818)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:143)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at io.trino.$gen.Trino_406_171_17____20240916_140731_2.call(Unknown Source)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
16-09-2024 07:34:49 PDT coral-test-workflow_selectHiveView INFO - 	at java.base/java.lang.Thread.run(Thread.java:840)

Current Behavior

  • HiveToRelConverter uses a properly configured HiveConf from Coral's HiveMetastoreClient wrapper.
  • CoralParseDriver creates a new, unconfigured HiveConf: 
    HiveConf hiveConf = new HiveConf();
  • This new HiveConf lacks necessary authentication and connection details.
  • Result: consistent authentication failures with execution-time/run-time errors like "Unable to determine current user".

Expected Behavior

  • All Coral components should use a consistent, properly configured HiveConf.
  • Authentication and Hive metastore data access should succeed consistently.

Impact

  • Query parsing/execution failures

Questions to Consider

  • Should we expose HiveConf getter/setter methods in Coral's HiveMetastoreClient wrapper (assuming Coral project wrapped around hmsClient to read-only by design)? What are the implications?
  • Is setting hive.support.sql11.reserved.keywords to false the right approach, or do we need a more flexible solution? Given the significant refactoring required, are there less invasive alternatives that could address this issue? How would these changes impact Coral's modularity, security, and maintainability?
@1fanwang 1fanwang added the help wanted Extra attention is needed label Sep 20, 2024
@1fanwang
Copy link
Collaborator Author

cc @wmoustafa @KevinGe00 @aastha25 @ljfgem @nagarathnam200

@1fanwang 1fanwang mentioned this issue Sep 23, 2024
Open
@KevinGe00 KevinGe00 mentioned this issue Sep 26, 2024
Merged
@1fanwang
Copy link
Collaborator Author

Thank you for fixing the issue @KevinGe00, really appreciate it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Always allow unquoted keywords as column names KevinGe00/coral
1 participant
@1fanwang