Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Simplify zip/comap, remove join from the implementation. #501

Closed
goodwanghan opened this issue Aug 7, 2023 · 0 comments · Fixed by #499
Closed

[FEATURE] Simplify zip/comap, remove join from the implementation. #501

goodwanghan opened this issue Aug 7, 2023 · 0 comments · Fixed by #499
Labels
core feature enhancement New feature or request
Milestone

Comments

@goodwanghan
Copy link
Collaborator

Currently zip and comap logic is convoluted. It also relies on join operation that is not well supported by Ray. So we need to use pure map and group map operations to realize this operation.

Actually zip and comap should always stay together, so in the long run we may need a breaking change to merge these two functions. If we can merge we may also get rid of the need of dataframe metadata, which is extremely hard to maintain.

But as the first step, we will clean up the logic and remove joins first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core feature enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant