Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] AQE being used on Databricks even when its disabled #6158

Closed
tgravescs opened this issue Jul 29, 2022 · 0 comments · Fixed by #6159
Closed

[BUG] AQE being used on Databricks even when its disabled #6158

tgravescs opened this issue Jul 29, 2022 · 0 comments · Fixed by #6159
Assignees
@tgravescs
Labels
bug Something isn't working P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

Describe the bug
We have a customer query that is failing on databricks with IllegalStateException: Custom columnar rules cannot transform exchange node to something else. and the stack trace shows its error during AQE when we have AQE disabled:

at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.newQueryStage(AdaptiveSparkPlanExec.scala:777)

This occurs when they are querying files stored in delta tables.
@tgravescs tgravescs added bug Something isn't working P0 Must have for release labels Jul 29, 2022
@tgravescs tgravescs self-assigned this Jul 29, 2022
This was referenced Jul 29, 2022
Merged
Open
@sameerz sameerz changed the title [BUG]AQE being used on Databricks even when its disabled [BUG] AQE being used on Databricks even when its disabled Aug 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tgravescs tgravescs

Labels
bug Something isn't working P0 Must have for release
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Workaround for Databricks using AQE even when disabled tgravescs/spark-rapids
1 participant
@tgravescs