Implementation of chunk balancer feature #1775
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CONTEXT: This feature uses empirical data about the timings of each MPI process to adjust the chunk sizes to achieve optimal load balancing for repeated simulations, such as iterations in an optimization run. Processes which are consistently slow and overworked will have their chunk sizes reduced for the next iteration of an optimization run, and underworked processes will handle larger chunks. This approach has the advantage that it implicitly incorporates variable load between different machines running the MPI processes. SCOPE: - Added abstract class for chunk balancer - Added DefaultChunkBalancer implementation, which adjusts chunk sizes according to per-process working time (ignoring all-all comms) while maintaining previous split directions - Wrote unit tests for DefaultChunkBalancer to check for improvement in load-balancing, convergence to a load-balanced state, and that split_pos values are adjusted correctly for each iteration. - Wrote unit tests for the MockSimulation class used by the DefaultChunkBalancerTests - Added a binary_partition_utils.py library which includes lots of tree traversal algorithms which are useful to have for the chunk balancer - Wrote unit tests for binary_partition_utils.py
Codecov Report
@@ Coverage Diff @@
## master #1775 +/- ##
==========================================
+ Coverage 74.41% 75.20% +0.79%
==========================================
Files 13 16 +3
Lines 4581 4796 +215
==========================================
+ Hits 3409 3607 +198
- Misses 1172 1189 +17
|
stevengj
reviewed
Sep 30, 2021
stevengj
reviewed
Sep 30, 2021
stevengj
reviewed
Sep 30, 2021
…ts between runs of a program
stevengj
reviewed
Oct 7, 2021
|
Failing tests? |
|
Sorry about that, made a minor change that broke a test on |
mawc2019
pushed a commit
to mawc2019/meep
that referenced
this pull request
Nov 3, 2021
* Implementation of chunk balancer feature CONTEXT: This feature uses empirical data about the timings of each MPI process to adjust the chunk sizes to achieve optimal load balancing for repeated simulations, such as iterations in an optimization run. Processes which are consistently slow and overworked will have their chunk sizes reduced for the next iteration of an optimization run, and underworked processes will handle larger chunks. This approach has the advantage that it implicitly incorporates variable load between different machines running the MPI processes. SCOPE: - Added abstract class for chunk balancer - Added DefaultChunkBalancer implementation, which adjusts chunk sizes according to per-process working time (ignoring all-all comms) while maintaining previous split directions - Wrote unit tests for DefaultChunkBalancer to check for improvement in load-balancing, convergence to a load-balanced state, and that split_pos values are adjusted correctly for each iteration. - Wrote unit tests for the MockSimulation class used by the DefaultChunkBalancerTests - Added a binary_partition_utils.py library which includes lots of tree traversal algorithms which are useful to have for the chunk balancer - Wrote unit tests for binary_partition_utils.py * fixed wrong test filename in Makefile.am * add new modules to Makefile.am to be included in __init__.py * bugfix in tests * more test bugfixes * avoid merge conflict * refactored class names, added documentation for adjusting chunk layouts between runs of a program * update to Parallel_Meep.md for missing chunk layout file * test bugfix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CONTEXT: This feature uses empirical data about the timings of each MPI process to adjust the chunk sizes to achieve optimal load balancing for repeated simulations, such as iterations in an optimization run. Processes which are consistently slow and overworked will have their chunk sizes reduced for the next iteration of an optimization run, and underworked processes will handle larger chunks. This approach has the advantage that it implicitly incorporates variable load between different machines running the MPI processes.
SCOPE: