Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta Lake compaction writes can fallback due to WriteIntoDeltaCommand #11169

Open
jlowe opened this issue Jul 11, 2024 · 0 comments
Open

Delta Lake compaction writes can fallback due to WriteIntoDeltaCommand #11169

jlowe opened this issue Jul 11, 2024 · 0 comments
Labels
feature request New feature or request

Comments

@jlowe
Copy link
Member

jlowe commented Jul 11, 2024

While testing the fix for #11167, I noticed that the writes for compaction on Databricks 13.3 are not executing on the GPU due to a new, unsupported operator appearing in the plan:

!Exec <DataWritingCommandExec> cannot run on GPU because not all data writing commands can be replaced
  ! <WriteIntoDeltaCommand> cannot run on GPU because GPU does not currently support the operator class com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand
  !Exec <WriteFilesExec> cannot run on GPU because WriteFilesExec can't run on GPU because parent can't run on GPU
@jlowe jlowe added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jul 11, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants