Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48961][PYTHON] Make the parameter naming of PySparkException consistent with JVM #47436

Closed
wants to merge 5 commits into from

Conversation

itholic
Copy link
Contributor

@itholic itholic commented Jul 22, 2024

What changes were proposed in this pull request?

This PR proposes to make the parameter naming of PySparkException consistent with JVM

Why are the changes needed?

The parameter names of PySparkException are different from SparkException so there is an inconsistency when searching those parameters from error logs.

SparkException:

class SparkException(
message: String,
cause: Throwable,
errorClass: Option[String],
messageParameters: Map[String, String],
context: Array[QueryContext] = Array.empty)
extends Exception(message, cause) with SparkThrowable {

PySparkException:

class PySparkException(Exception):
"""
Base Exception for handling errors generated from PySpark.
"""
def __init__(
self,
message: Optional[str] = None,
error_class: Optional[str] = None,
message_parameters: Optional[Dict[str, str]] = None,
query_contexts: Optional[List["QueryContext"]] = None,
):

Does this PR introduce any user-facing change?

The error parameter names are changed from:

  • error_class -> errorClass
  • message_parameters -> messageParameters
  • query_contexts -> context

How was this patch tested?

The existing CI should pass

Was this patch authored or co-authored using generative AI tooling?

No.

@itholic
Copy link
Contributor Author

itholic commented Jul 22, 2024

cc @HyukjinKwon

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if tests pass

@itholic itholic closed this in 090ad9f Jul 24, 2024
@itholic
Copy link
Contributor Author

itholic commented Jul 24, 2024

Merged into master. Thanks!

ilicmarkodb pushed a commit to ilicmarkodb/spark that referenced this pull request Jul 29, 2024
… consistent with JVM

### What changes were proposed in this pull request?

This PR proposes to make the parameter naming of `PySparkException` consistent with JVM

### Why are the changes needed?

The parameter names of `PySparkException` are different from `SparkException` so there is an inconsistency when searching those parameters from error logs.

SparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/common/utils/src/main/scala/org/apache/spark/SparkException.scala#L27-L33

PySparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/python/pyspark/errors/exceptions/base.py#L29-L40

### Does this PR introduce _any_ user-facing change?

The error parameter names are changed from:
- `error_class` -> `errorClass`
- `message_parameters` -> `messageParameters`
- `query_contexts` -> `context`

### How was this patch tested?

The existing CI should pass

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47436 from itholic/SPARK-48961.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Haejoon Lee <haejoon.lee@databricks.com>
fusheng-rd pushed a commit to fusheng-rd/spark that referenced this pull request Aug 6, 2024
… consistent with JVM

### What changes were proposed in this pull request?

This PR proposes to make the parameter naming of `PySparkException` consistent with JVM

### Why are the changes needed?

The parameter names of `PySparkException` are different from `SparkException` so there is an inconsistency when searching those parameters from error logs.

SparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/common/utils/src/main/scala/org/apache/spark/SparkException.scala#L27-L33

PySparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/python/pyspark/errors/exceptions/base.py#L29-L40

### Does this PR introduce _any_ user-facing change?

The error parameter names are changed from:
- `error_class` -> `errorClass`
- `message_parameters` -> `messageParameters`
- `query_contexts` -> `context`

### How was this patch tested?

The existing CI should pass

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47436 from itholic/SPARK-48961.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Haejoon Lee <haejoon.lee@databricks.com>
attilapiros pushed a commit to attilapiros/spark that referenced this pull request Oct 4, 2024
… consistent with JVM

### What changes were proposed in this pull request?

This PR proposes to make the parameter naming of `PySparkException` consistent with JVM

### Why are the changes needed?

The parameter names of `PySparkException` are different from `SparkException` so there is an inconsistency when searching those parameters from error logs.

SparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/common/utils/src/main/scala/org/apache/spark/SparkException.scala#L27-L33

PySparkException:
https://github.com/apache/spark/blob/6508b1f5e18731359354af0a7bcc1382bc4f356b/python/pyspark/errors/exceptions/base.py#L29-L40

### Does this PR introduce _any_ user-facing change?

The error parameter names are changed from:
- `error_class` -> `errorClass`
- `message_parameters` -> `messageParameters`
- `query_contexts` -> `context`

### How was this patch tested?

The existing CI should pass

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47436 from itholic/SPARK-48961.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Haejoon Lee <haejoon.lee@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants