Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48618][SQL] Utilize the ErrorCode and SQLState returned in SQLException to make errors more accurate #46969

Closed
wants to merge 4 commits into from

Conversation

panbingkun
Copy link
Contributor

@panbingkun panbingkun commented Jun 13, 2024

What changes were proposed in this pull request?

The pr aims to use the ErrorCode and SQLState returned in SQLException to make errors more accurate.

Why are the changes needed?

  • eg for mysql

    withTable(s"$catalogName.tbl1", s"$catalogName.tbl2") {
          sql(s"CREATE TABLE $catalogName.tbl1 (col1 INT, col2 INT)")
          sql(s"CREATE TABLE $catalogName.tbl2 (col1 INT, col2 INT)")
          sql(s"ALTER TABLE $catalogName.tbl2 RENAME TO tbl1")
        }
    
  • Before:

    [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table renaming from tbl2 to tbl1 SQLSTATE: 
    HV000
    org.apache.spark.sql.AnalysisException: [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table 
    renaming from tbl2 to tbl1 SQLSTATE: HV000
    	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:767)
        at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:751)
        at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:348)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
        at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
        at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
    
  • After:

    [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
    Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing 
    objects. SQLSTATE: 42P07
    org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException: [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
    Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects. SQLSTATE: 42P07
        at org.apache.spark.sql.errors.QueryCompilationErrors$.tableAlreadyExistsError(QueryCompilationErrors.scala:2643)
        at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:343)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
        at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
        at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
    
  • Aligning similar logic to PostgresDialect

    } else if (errorClass == "FAILED_JDBC.RENAME_TABLE") {
    val newTable = messageParameters("newName")
    throw QueryCompilationErrors.tableAlreadyExistsError(newTable)

Does this PR introduce any user-facing change?

Yes, End users will accurately know the cause of the error.

How was this patch tested?

Add new UT.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the SQL label Jun 13, 2024
case sqlException: SQLException =>
logSQLException(sqlException, errorClass, messageParameters, description)
sqlException.getErrorCode match {
case 955 if errorClass == "FAILED_JDBC.RENAME_TABLE" =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref:

errorCode: 955, sqlState: 42000, message: ORA-00955: name is already used by an existing object
https://docs.oracle.com/error-help/db/ora-00955/, errorClass: FAILED_JDBC.RENAME_TABLE, messageParameters: Map(url -> jdbc:oracle:thin:system/Th1s1sThe0racle#Pass@//10.1.0.20:38407/freepdb1, oldName -> `tbl2`, newName -> `tbl1`),description: Failed table renaming from tbl2 to tbl1

sqlException.getErrorCode match {
case 3729 =>
throw NonEmptyNamespaceException(
namespace = messageParameters.get("namespace").toArray,
details = sqlException.getMessage,
cause = Some(e))
case 15335 if errorClass == "FAILED_JDBC.RENAME_TABLE" =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref:

errorCode: 15335, sqlState: S0001, message: Error: The new name 'tbl1' is already in use as a object name and would cause a duplicate that is not permitted., errorClass: FAILED_JDBC.RENAME_TABLE, messageParameters: Map(url -> jdbc:sqlserver://10.1.0.20:38701;user=sa;***;encrypt=true;trustServerCertificate=true, oldName -> `tbl2`, newName -> `tbl1`),description: Failed table renaming from tbl2 to tbl1

sqlException.getSQLState match {
// https://www.ibm.com/docs/en/db2/11.5?topic=messages-sqlstate
case "42893" =>
throw NonEmptyNamespaceException(
namespace = messageParameters.get("namespace").toArray,
details = sqlException.getMessage,
cause = Some(e))
case "42710" if errorClass == "FAILED_JDBC.RENAME_TABLE" =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref:

errorCode: -601, sqlState: 42710, message: The name of the object to be created is identical to the existing name "DB2INST1.tbl1" of type "TABLE".. SQLCODE=-601, SQLSTATE=42710, DRIVER=4.33.31, errorClass: FAILED_JDBC.RENAME_TABLE, messageParameters: Map(url -> jdbc:db2://10.1.0.20:43213/foo:user=db2inst1;***;retrieveMessagesFromServerOnGetMessage=true;, oldName -> `tbl2`, newName -> `tbl1`),description: Failed table renaming from tbl2 to tbl1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@panbingkun panbingkun changed the title [WIP][SQL] fine-grained jdbc error [SPARK-48618][SQL] Use the ErrorCode and SQLState returned in SQLException to make errors more accurate Jun 13, 2024
@panbingkun panbingkun marked this pull request as ready for review June 13, 2024 11:04
@panbingkun
Copy link
Contributor Author

panbingkun commented Jun 13, 2024

cc @cloud-fan @yaooqinn @HyukjinKwon

@panbingkun panbingkun changed the title [SPARK-48618][SQL] Use the ErrorCode and SQLState returned in SQLException to make errors more accurate [SPARK-48618][SQL] Utilize the ErrorCode and SQLState returned in SQLException to make errors more accurate Jun 13, 2024
@panbingkun
Copy link
Contributor Author

also cc @MaxGekk

@MaxGekk
Copy link
Member

MaxGekk commented Aug 14, 2024

+1, LGTM. Merging to master.
Thank you, @panbingkun.

@MaxGekk MaxGekk closed this in 7e76188 Aug 14, 2024
IvanK-db pushed a commit to IvanK-db/spark that referenced this pull request Sep 20, 2024
… SQLException to make errors more accurate

### What changes were proposed in this pull request?
The pr aims to use the `ErrorCode` and `SQLState` returned in SQLException to make errors more accurate.

### Why are the changes needed?
- eg for `mysql`
  ```
  withTable(s"$catalogName.tbl1", s"$catalogName.tbl2") {
        sql(s"CREATE TABLE $catalogName.tbl1 (col1 INT, col2 INT)")
        sql(s"CREATE TABLE $catalogName.tbl2 (col1 INT, col2 INT)")
        sql(s"ALTER TABLE $catalogName.tbl2 RENAME TO tbl1")
      }
  ```
- Before:
  ```
  [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table renaming from tbl2 to tbl1 SQLSTATE:
  HV000
  org.apache.spark.sql.AnalysisException: [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table
  renaming from tbl2 to tbl1 SQLSTATE: HV000
  	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:767)
	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:751)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:348)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- After:
  ```
  [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing
  objects. SQLSTATE: 42P07
  org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException: [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects. SQLSTATE: 42P07
	  at org.apache.spark.sql.errors.QueryCompilationErrors$.tableAlreadyExistsError(QueryCompilationErrors.scala:2643)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:343)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- Aligning similar logic to `PostgresDialect`
https://github.com/apache/spark/blob/08e741b92b8fc9e43c838d0849317916218414ce/sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala#L273-L275

### Does this PR introduce _any_ user-facing change?
Yes, End users will accurately know the cause of the error.

### How was this patch tested?
Add new UT.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46969 from panbingkun/fine-grained_jdbc_error.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
attilapiros pushed a commit to attilapiros/spark that referenced this pull request Oct 4, 2024
… SQLException to make errors more accurate

### What changes were proposed in this pull request?
The pr aims to use the `ErrorCode` and `SQLState` returned in SQLException to make errors more accurate.

### Why are the changes needed?
- eg for `mysql`
  ```
  withTable(s"$catalogName.tbl1", s"$catalogName.tbl2") {
        sql(s"CREATE TABLE $catalogName.tbl1 (col1 INT, col2 INT)")
        sql(s"CREATE TABLE $catalogName.tbl2 (col1 INT, col2 INT)")
        sql(s"ALTER TABLE $catalogName.tbl2 RENAME TO tbl1")
      }
  ```
- Before:
  ```
  [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table renaming from tbl2 to tbl1 SQLSTATE:
  HV000
  org.apache.spark.sql.AnalysisException: [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table
  renaming from tbl2 to tbl1 SQLSTATE: HV000
  	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:767)
	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:751)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:348)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- After:
  ```
  [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing
  objects. SQLSTATE: 42P07
  org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException: [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects. SQLSTATE: 42P07
	  at org.apache.spark.sql.errors.QueryCompilationErrors$.tableAlreadyExistsError(QueryCompilationErrors.scala:2643)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:343)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- Aligning similar logic to `PostgresDialect`
https://github.com/apache/spark/blob/08e741b92b8fc9e43c838d0849317916218414ce/sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala#L273-L275

### Does this PR introduce _any_ user-facing change?
Yes, End users will accurately know the cause of the error.

### How was this patch tested?
Add new UT.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46969 from panbingkun/fine-grained_jdbc_error.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
himadripal pushed a commit to himadripal/spark that referenced this pull request Oct 19, 2024
… SQLException to make errors more accurate

### What changes were proposed in this pull request?
The pr aims to use the `ErrorCode` and `SQLState` returned in SQLException to make errors more accurate.

### Why are the changes needed?
- eg for `mysql`
  ```
  withTable(s"$catalogName.tbl1", s"$catalogName.tbl2") {
        sql(s"CREATE TABLE $catalogName.tbl1 (col1 INT, col2 INT)")
        sql(s"CREATE TABLE $catalogName.tbl2 (col1 INT, col2 INT)")
        sql(s"ALTER TABLE $catalogName.tbl2 RENAME TO tbl1")
      }
  ```
- Before:
  ```
  [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table renaming from tbl2 to tbl1 SQLSTATE:
  HV000
  org.apache.spark.sql.AnalysisException: [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table
  renaming from tbl2 to tbl1 SQLSTATE: HV000
  	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:767)
	  at org.apache.spark.sql.jdbc.JdbcDialect.classifyException(JdbcDialects.scala:751)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:348)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- After:
  ```
  [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing
  objects. SQLSTATE: 42P07
  org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException: [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `tbl1` because it already exists.
  Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects. SQLSTATE: 42P07
	  at org.apache.spark.sql.errors.QueryCompilationErrors$.tableAlreadyExistsError(QueryCompilationErrors.scala:2643)
	  at org.apache.spark.sql.jdbc.MySQLDialect.classifyException(MySQLDialect.scala:343)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.classifyException(JdbcUtils.scala:1271)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1(JDBCTableCatalog.scala:125)
	  at org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog.$anonfun$renameTable$1$adapted(JDBCTableCatalog.scala:116)
	  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.withConnection(JdbcUtils.scala:1279)
  ```

- Aligning similar logic to `PostgresDialect`
https://github.com/apache/spark/blob/08e741b92b8fc9e43c838d0849317916218414ce/sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala#L273-L275

### Does this PR introduce _any_ user-facing change?
Yes, End users will accurately know the cause of the error.

### How was this patch tested?
Add new UT.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46969 from panbingkun/fine-grained_jdbc_error.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants