Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Implement all partitioning strategies for Dask #500

Closed
goodwanghan opened this issue Aug 7, 2023 · 0 comments · Fixed by #499
Closed

[FEATURE] Implement all partitioning strategies for Dask #500

goodwanghan opened this issue Aug 7, 2023 · 0 comments · Fixed by #499
Labels
dask enhancement New feature or request
Milestone
0.8.7

Comments

@goodwanghan
Copy link
Collaborator

Currently, Dask partitioning implementation is very incomplete. It mostly relies on Dask's own partitioning logic, and it only supports even partitioning when there is no partition keys, and the approach is not scalable either. So we need to implement all partitioning algos: "hash", "rand" and "even" for cases with or without partition keys.
@goodwanghan goodwanghan added enhancement New feature or request dask labels Aug 7, 2023
@goodwanghan goodwanghan added this to the 0.8.7 milestone Aug 7, 2023
@goodwanghan goodwanghan linked a pull request Aug 7, 2023 that will close this issue
Implement all partitioning algos for dask, remove join from zip for all backends #499
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dask enhancement New feature or request
Projects
None yet
Milestone
0.8.7
Development

Successfully merging a pull request may close this issue.

Implement all partitioning algos for dask, remove join from zip for all backends fugue-project/fugue
1 participant
@goodwanghan