You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
But when even partitioning is needed, in most cases the input dataframe is very small because even partitioning is commonly used in small data large compute. So a trade-off is that we can compute the dataframe as a pandas dataframe, and reconstruct a dask dataframe to partition evenly.
As phase one, we only support even partition on small data. On large data, it may have scalability issues
The text was updated successfully, but these errors were encountered:
Currently even partitioning doesn't take effect on Dask dataframes.
And this issue was not resolved on Dask side.
But when even partitioning is needed, in most cases the input dataframe is very small because even partitioning is commonly used in small data large compute. So a trade-off is that we can compute the dataframe as a pandas dataframe, and reconstruct a dask dataframe to partition evenly.
As phase one, we only support even partition on small data. On large data, it may have scalability issues
The text was updated successfully, but these errors were encountered: