Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48974][SQL][SS][ML][MLLIB] Use SparkSession.implicits instead of SQLContext.implicits #47457

Closed

Conversation

LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Jul 23, 2024

What changes were proposed in this pull request?

This PR replaces SQLContext.implicits with SparkSession.implicits in the Spark codebase.

Why are the changes needed?

Reduce the usage of code from SQLContext within the internal code of Spark.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

@LuciferYang LuciferYang changed the title [WIP] Use SparkSession.implicits instead of SQLContext.implicits [WIP][SQL][SS][ML][MLLIB] Use SparkSession.implicits instead of SQLContext.implicits Jul 23, 2024
@LuciferYang LuciferYang changed the title [WIP][SQL][SS][ML][MLLIB] Use SparkSession.implicits instead of SQLContext.implicits [SPARK-48974][SQL][SS][ML][MLLIB] Use SparkSession.implicits instead of SQLContext.implicits Jul 23, 2024
@LuciferYang
Copy link
Contributor Author

cc @HyukjinKwon
Do you think we can mark org.apache.spark.sql.SQLContext.implicits as deprecated?

@LuciferYang
Copy link
Contributor Author

Merged into master. Thanks @HyukjinKwon

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @LuciferYang and @HyukjinKwon .

@LuciferYang
Copy link
Contributor Author

Thank you @dongjoon-hyun ~

ilicmarkodb pushed a commit to ilicmarkodb/spark that referenced this pull request Jul 29, 2024
…d of `SQLContext.implicits`

### What changes were proposed in this pull request?
This PR replaces `SQLContext.implicits`  with `SparkSession.implicits` in the Spark codebase.

### Why are the changes needed?
Reduce the usage of code from `SQLContext` within the internal code of Spark.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47457 from LuciferYang/use-sparksession-implicits.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
fusheng-rd pushed a commit to fusheng-rd/spark that referenced this pull request Aug 6, 2024
…d of `SQLContext.implicits`

### What changes were proposed in this pull request?
This PR replaces `SQLContext.implicits`  with `SparkSession.implicits` in the Spark codebase.

### Why are the changes needed?
Reduce the usage of code from `SQLContext` within the internal code of Spark.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47457 from LuciferYang/use-sparksession-implicits.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
attilapiros pushed a commit to attilapiros/spark that referenced this pull request Oct 4, 2024
…d of `SQLContext.implicits`

### What changes were proposed in this pull request?
This PR replaces `SQLContext.implicits`  with `SparkSession.implicits` in the Spark codebase.

### Why are the changes needed?
Reduce the usage of code from `SQLContext` within the internal code of Spark.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47457 from LuciferYang/use-sparksession-implicits.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants