Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plot process cpu affinity (psutil) #134

Closed

Conversation

jjexpat00
Copy link
Contributor

@jjexpat00 jjexpat00 commented May 7, 2021

This implementation of psutil.Process().cpu_affinity() lets the user parameterize a selection of cores/threads specific to their plotting system and bind configured CPU affinity for newly created plotting processes. As specified in the readme and config.yaml documentation, this functionality gives the user greater control over their plotting machine's resources.

Some users run full node, wallet, and harvester in addition to plotting processes - I have personally experienced and have seen others run into issues when CPU resource allocation is fully consumed by plotting processes. Even on high-end multi-core CPU's, this may lead to node de-sync and delayed plot scanning (to name a few).

Enabling CPU core/thread restrictions will objectively decrease plotting speed, making this feature optional and disabled by default.

Let me know if there are better formatting conventions and logic simplification.

Any suggestions are welcome.

@trekb
Copy link

trekb commented May 8, 2021

Onother way of acomplish the same goal is to define the process priority. If the plotter process is launched with LOW priority, the system behaviour will be much better, while potentially still using all the available resources for plotting.

@feldsam
Copy link
Contributor

feldsam commented May 9, 2021

@trekb , there is already renice to priority 10 on linux machines and some "normal" priority on windows machines.

@stubmirror
Copy link

would be nice if it was a per job setting instead of global

@feldsam
Copy link
Contributor

feldsam commented May 11, 2021

@stubmirror contributions are welcome ;)

@swar swar changed the base branch from main to cpu-affinity May 13, 2021 02:51
@swar
Copy link
Owner

swar commented May 13, 2021

I want to add this to the development branch but with the amount of questions I have been getting lately, I need to find a way to make it so people don't start complaining why their plots are going slow and then realizing they used this functionality wrong. Will keep this open. I made a branch to merge my current changes with it as well.

@swar swar closed this May 13, 2021
@swar swar deleted the branch swar:cpu-affinity May 13, 2021 03:49
@swar swar reopened this May 13, 2021
@swar
Copy link
Owner

swar commented May 13, 2021

okay moved it to job level instead of global. should be good to go esp since its default set to false

@swar swar closed this May 13, 2021
This was referenced May 13, 2021
@MMJ997
Copy link

MMJ997 commented Jun 9, 2021

https://github.com/swar/Swar-Chia-Plot-Manager/tree/jjexpat00-psutil-plot-process-affinity
Two CPUs will have an error.

enable_cpu_affinity: false is also useless, the software will only call CPU0, not CPU1.
微信图片_20210609123722
微信图片_202106091237221
微信图片_202106091237222

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants