Match concurrency to available CPU bandwidth #2300
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maxim-kuvyrkov
force-pushed
the
limit-on-psi
branch
5 times, most recently
from
b78e715
to
940ced7
Compare
This change allows ninja to throttle number of parallel tasks based on feedback from Linux kernel's PSI (Pressure Stall Information) interfacts. It extends "-l" parameter to accept negative values; "-l-NN" means that ninja should limit concurrency when processes in current cgroup spend more than NN% of their time stalled on CPU. E.g., running "ninja -j100 -l-10" on a 32-core machine will quickly settle on parallelism of 32-34. This option is designed to make ninja use all CPU bandwidth available to a cgroup-based container, while not starting excessive number of processes, which could eat up all RAM. The motivation for this feature is to automatically reduce parallelism when the system is about to run out of RAM. If the system has swap enabled, "ninja -l-10" will dance with parallelism on the edge of just using a bit of swap. As soon as a process starts swapping, CPU "stalled" cycles increase, and parallelism is reduced. The same argument works when a process is waiting for its turn to use IO and/or network.
maxim-kuvyrkov
force-pushed
the
limit-on-psi
branch
from
940ced7
to
92878c3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change allows ninja to throttle number of parallel tasks based on feedback from Linux kernel's PSI (Pressure Stall Information) interfacts. It extends "-l" parameter to accept negative values; "-l-NN" means that ninja should limit concurrency when processes in current cgroup spend more than NN% of their time stalled on CPU.
E.g., running "ninja -j100 -l-10" on a 32-core machine will quickly settle on parallelism of 32-34.
This option is designed to make ninja use all CPU bandwidth available to a cgroup-based container, while not starting excessive number of processes, which could eat up all RAM.
The motivation for this feature is too automatically reduce parallelism when the system is about to run out of RAM. If the system has swap enabled, "ninja -l-10" will dance with parallelism on the edge of just using a bit of swap. As soon as a process starts swapping, that increases CPU "stalled" cycles, and parallelism is reduced. The same argument works when a process is waiting for its turn to use IO and/or network.