[WIP][SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting #47423
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 12, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala
Show resolved
Hide resolved
# Conflicts: # sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala # sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala # sql/core/src/test/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNodeSuite.scala # sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerWithSparkContextSuite.scala
davidm-db
reviewed
Aug 13, 2024
sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseLexer.g4
Outdated
Show resolved
Hide resolved
davidm-db
reviewed
Aug 13, 2024
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala
Outdated
Show resolved
Hide resolved
# Conflicts: # sql/core/src/test/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreterSuite.scala
MaxGekk
reviewed
Aug 15, 2024
| @@ -119,11 +120,13 @@ class AstBuilder extends DataTypeAstBuilder | |||
| override def visitCompoundOrSingleStatement( | |||
| ctx: CompoundOrSingleStatementContext): CompoundBody = withOrigin(ctx) { | |||
| Option(ctx.singleCompoundStatement()).map { s => | |||
| if (!SQLConf.get.sqlScriptingEnabled) { | |||
There was a problem hiding this comment.
Why don't you get the current config value from the active config obj conf? See other examples in the same file like conf.getConf(SQLConf.ENABLE_DEFAULT_COLUMNS)
There was a problem hiding this comment.
This will be rebased on: #47609. Can you please leave comments there first, and then we can get back to this one.
| origin = origin, | ||
| errorClass = "UNSUPPORTED_FEATURE.SQL_SCRIPTING_NOT_ENABLED", | ||
| cause = null, | ||
| messageParameters = Map("sqlScriptingEnabled" -> SQLConf.SQL_SCRIPTING_ENABLED.key)) |
There was a problem hiding this comment.
Please, follow the existing convention and quote the SQL config name using toSQLConf()
| @@ -269,7 +269,7 @@ org.apache.spark.sql.catalyst.parser.ParseException | |||
| "errorClass" : "PARSE_SYNTAX_ERROR", | |||
| "sqlState" : "42601", | |||
| "messageParameters" : { | |||
| "error" : "'.'", | |||
| "error" : "end of input", | |||
There was a problem hiding this comment.
Could you explain why this PR changes this error parameter?
There was a problem hiding this comment.
I will check this one
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This pull request introduces the logic of error handling inside SQL Scripting language. Now, it is possible to:
Why are the changes needed?
The intent is to add the possibility for user to handle SQL errors in a custom defined way..
Does this PR introduce any user-facing change?
No.
How was this patch tested?
There are already existing test suites for SQL scripting that have been improved to test new functionalities:
SqlScriptingParserSuiteSqlScriptingExecutionNodeSuiteSqlScriptingInterpreterSuiteWas this patch authored or co-authored using generative AI tooling?
No.
Additional changes:
Added
cleanupmethod insideCompoundBodyto handle premature exit. Currently this can happen while executingleavestatement.NEED TO FIX (follow-up work):