-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-26502][SQL] Move hiveResultString() from QueryExecution to HiveResult #23409
Conversation
@hvanhovell I tried to move it to a test class but |
Test build #100538 has finished for PR 23409 at commit
|
sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala
Outdated
Show resolved
Hide resolved
@MaxGekk . Could you revise the PR title a little bit? |
Test build #100543 has finished for PR 23409 at commit
|
sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala
Outdated
Show resolved
Hide resolved
sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala
Outdated
Show resolved
Hide resolved
sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala
Outdated
Show resolved
Hide resolved
sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala
Outdated
Show resolved
Hide resolved
import org.apache.spark.sql.execution.command.{DescribeTableCommand, ExecutedCommandExec, ShowTablesCommand} | ||
import org.apache.spark.sql.types._ | ||
|
||
object HiveResult { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HiveResult.hiveResultString seems could used for other(eg. MySql).So I suggest modify the name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can be used from different places. The main purpose of these methods is to return results in Hive-like form.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, let's stick with HiveResult.
# Conflicts: # sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
Test build #100618 has finished for PR 23409 at commit
|
Test build #100616 has finished for PR 23409 at commit
|
@hvanhovell The last commit 948414a can be merged. |
jenkins, retest this, please |
Test build #100660 has finished for PR 23409 at commit
|
import org.apache.spark.sql.internal.SQLConf | ||
import org.apache.spark.sql.types._ | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, it's just moving Hive stuff from sql
to sql
. If it were getting rid of it completely or moving it to test scope only, then I'm supportive but now it just move the function.
Kind of neutral on this. Code itself looks good. If any committer pushes it within few days, then it's good. Otherwise, I would close.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, it's just moving Hive stuff from sql to sql.
This is not just moving it inside of sql. The idea is to clean up common class QueryExecution
from pretty specific method which is used in tests mostly (and in SparkSQLDriver
unfortunately).
If it were getting rid of it completely or moving it to test scope only, then I'm supportive but now it just move the function.
I tried all those variants like:
- Moving to
sql/core/test
but it seems weird thathive-thriftserver
(SparkSQLDriver
) depends on a test jar. - Moving to
hive
since hive's tests use the method makes cycle dependencieshive
<->sql/core
.
It would be nice if SQLQueryTestSuite
doesn't depend on Hive related method. Is it really needed?
Otherwise, I would close.
@hvanhovell Please, protect ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…eResult ## What changes were proposed in this pull request? In the PR, I propose to move `hiveResultString()` out of `QueryExecution` and put it to a separate object. Closes apache#23409 from MaxGekk/hive-result-string. Lead-authored-by: Maxim Gekk <maxim.gekk@databricks.com> Co-authored-by: Maxim Gekk <max.gekk@gmail.com> Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>
…eResult In the PR, I propose to move `hiveResultString()` out of `QueryExecution` and put it to a separate object. Closes apache#23409 from MaxGekk/hive-result-string. Lead-authored-by: Maxim Gekk <maxim.gekk@databricks.com> Co-authored-by: Maxim Gekk <max.gekk@gmail.com> Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>
What changes were proposed in this pull request?
In the PR, I propose to move
hiveResultString()
out ofQueryExecution
and put it to a separate object.