Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TypeCoercion optimizer rule #3221

Closed
Tracked by #2355
andygrove opened this issue Aug 22, 2022 · 2 comments
Closed
Tracked by #2355

Add TypeCoercion optimizer rule #3221

andygrove opened this issue Aug 22, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We currently do a lot of type coercion in the physical planning phase. I would like the ability to have type coercion and validation happen in the logical plan.

This would benefit projects such as Dask SQL that are using the SQL query planner.

Describe the solution you'd like
Add a new TypeCoercion rule that adds explicit CASTs where appropriate.

Describe alternatives you've considered
None

Additional context
None

@liukun4515
Copy link
Contributor

@andygrove
I go through the spark code about the type coercion and find https://github.com/apache/spark/blob/55c3347c48f93a9c5c5c2fb00b30f838eb081b7f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L323

Spark has put the type coercion to the analysis phase, but DF don't have this phase.

@andygrove
Copy link
Member Author

This was implemented in #3222

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants