Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unify split_by_effort and split_by_cost #1498

Closed
stevengj opened this issue Feb 10, 2021 · 1 comment · Fixed by #1499
Closed

unify split_by_effort and split_by_cost #1498

stevengj opened this issue Feb 10, 2021 · 1 comment · Fixed by #1499

Comments

@stevengj
Copy link
Collaborator

stevengj commented Feb 10, 2021

Steps:

  1. Change split_by_cost to not split by prime factors anymore — instead, like split_by_effort, always divide into two pieces along some axis, except that the "halves" are unequal (weighed by the number of processes that will go into each half) for an odd number of processes. (Recursively.) (Hence it will always give the requested number of chunks, regardless of the prime factorization.)

  2. Give split_by_cost an option to use a simpler cost metric of just the number of pixels in a subvolume, similar to split_by_effort.

At this point, split_by_effort should be redundant (and slower) and can be eliminated.

@oskooi
Copy link
Collaborator

oskooi commented Feb 11, 2021

Just as a comparison, the chunk layouts for split_by_effort versus split_by_cost for a square 2d cell with just vacuum shows that currently they often produce different results:

chunk_layout_2d

The results for split_by_effort for the case of 5 and 10 chunks are clearly better than split_by_cost since those chunks have less surface area.

Also note that even when the results are similar (for the case of 8 chunks), the ordering of the chunks is different as demonstrated by their colors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants