Skip to content

Commit

Permalink
[SPARK-41532][CONNECT][FOLLOWUP] Make the scala client using the same…
Browse files Browse the repository at this point in the history
… error class as python client

### What changes were proposed in this pull request?
The python connect client define the error class in `error_classes.py`.
`SESSION_NOT_SAME` is an error class used to check the `SparkSession` of one dataset is the same the other dataset. Please refer [`error_classes.py` ](https://github.com/apache/spark/blob/546e39c5dabc1111243ab81b6238dc893d9993e0/python/pyspark/errors/error_classes.py#L678C1-L678C1)
But the scala connect client not the the same error class.

### Why are the changes needed?
This PR make the scala client using the same error class as python client.

### Does this PR introduce _any_ user-facing change?
'No'.
Just update the inner implementation.

### How was this patch tested?
Exists test cases.

Closes apache#42256 from beliefer/SPARK-41532_followup.

Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
  • Loading branch information
beliefer authored and ragnarok56 committed Mar 2, 2024
1 parent c72645c commit 9497342
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 1 deletion.
5 changes: 5 additions & 0 deletions common/utils/src/main/resources/error/error-classes.json
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,11 @@
"message" : [
"Error instantiating Spark Connect plugin: <msg>"
]
},
"SESSION_NOT_SAME" : {
"message" : [
"Both Datasets must belong to the same SparkSession."
]
}
}
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1750,7 +1750,10 @@ class Dataset[T] private[sql] (

private def checkSameSparkSession(other: Dataset[_]): Unit = {
if (this.sparkSession.sessionId != other.sparkSession.sessionId) {
throw new SparkException("Both Datasets must belong to the same SparkSession")
throw new SparkException(
errorClass = "CONNECT.SESSION_NOT_SAME",
messageParameters = Map.empty,
cause = null)
}
}

Expand Down

0 comments on commit 9497342

Please sign in to comment.