Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] GpuGlobalLimitExec and GpuCollectLimitExec support offset #5589

Closed
firestarman opened this issue May 23, 2022 · 2 comments · Fixed by #5946
Closed

[FEA] GpuGlobalLimitExec and GpuCollectLimitExec support offset #5589

firestarman opened this issue May 23, 2022 · 2 comments · Fixed by #5946
Assignees
Labels
audit_3.4.0 Audit related tasks for 3.4.0 feature request New feature or request

Comments

@firestarman
Copy link
Collaborator

firestarman commented May 23, 2022

A new parameter named offset is introduced in GlobalLimitExec and CollectLimitExec from Spark 340. Accordingly, our GPU versions should also support it.

case class GlobalLimitExec(limit: Int = -1, child: SparkPlan, offset: Int = 0)
  extends BaseLimitExec {
  assert(limit >= 0 || (limit == -1 && offset > 0))
  ...
}

Now they will fall back to CPU for non-zero offset.

@firestarman firestarman added feature request New feature or request ? - Needs Triage Need team to review and classify labels May 23, 2022
@firestarman firestarman added the audit_3.4.0 Audit related tasks for 3.4.0 label May 23, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 24, 2022
@firestarman firestarman changed the title [FEA] GpuGlobalLimitExec supports offset [FEA] GpuGlobalLimitExec and GpuCollectLimitExec support offset May 25, 2022
@revans2
Copy link
Collaborator

revans2 commented May 25, 2022

For 3.4.0 we have to either implement this or fall back to the CPU if offset is not 0.

@jlowe
Copy link
Contributor

jlowe commented May 25, 2022

Fallback to CPU should be handled in 5cad5b7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audit_3.4.0 Audit related tasks for 3.4.0 feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants