New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Spark EVEN REPARTITION doesn't work for keys that are not at the first positions #119

Closed

goodwanghan opened this issue Dec 1, 2020 · 0 comments · Fixed by #118

Labels

bug spark

Milestone

Collaborator

goodwanghan commented Dec 1, 2020

Minimal Code To Reproduce

with FugueWorkflow() as dag:
   df = dag.df([[0,"a",1],[1,"b",2]], "z:int,a:str,b:int")
   df.partition(algo="even", by=["b","z"]).transform()

The problem is after even repartition, the column orders are messed up

goodwanghan added bug spark labels

goodwanghan added this to the 0.4.8 milestone

goodwanghan linked a pull request

that will close this issue

release 0.4.8 #118

Merged

goodwanghan closed this as completed in #118

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment