Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpulimit is "stopping" the script, but it continue in the background and send data to the stdout #91

Open
vstoykovbg opened this issue Nov 22, 2019 · 5 comments

Comments

@vstoykovbg
Copy link

vstoykovbg commented Nov 22, 2019

Here is how it looks like: https://www.youtube.com/watch?v=LN_DjgREXjU

CPUlimit version 2.4

@abhimishra01
Copy link

++ We have been facing exactly same issue in our implementation

Would request the maintainers to provide some light on this issue

@abhimishra01
Copy link

++ We have been facing exactly same issue in our implementation

Would request the maintainers to provide some light on this issue

@opsengine @urdh @HelderMagalhaes @praetp

@HelderMagalhaes
Copy link
Contributor

Here is how it looks like: https://www.youtube.com/watch?v=LN_DjgREXjU

CPUlimit version 2.4

Took a look at the video – it is the approach which seems quite flawed:

  1. Searching for script name in process list - if the process forks, it's no longer assured command line (and even process name) will be kept.
  2. The script is polling, meaning you may skip a number of events which occur in between "scans" (i.e., aliasing).

😐

@HelderMagalhaes
Copy link
Contributor

++ We have been facing exactly same issue in our implementation

Would request the maintainers to provide some light on this issue

@opsengine @urdh @HelderMagalhaes @praetp

Not a maintainer here but claiming "same issue" is clearly not enough: I'd suggest attaching a clear test case which reproduces (and take a look at the previous comment as input).

🤔

@abhimishra01
Copy link

++ We have been facing exactly same issue in our implementation
Would request the maintainers to provide some light on this issue

@opsengine @urdh @HelderMagalhaes @praetp

Not a maintainer here but claiming "same issue" is clearly not enough: I'd suggest attaching a clear test case which reproduces (and take a look at the previous comment as input).

🤔

I will try to see how I can create a workflow that can replicate this in someone else's setup as I cannot share the exact data engineering code which is confidential.
Quick Outline of what I ve been doing :-

  • Created a cpu-monitor service in linux machine utilizing the cpulimit tool
  • It fetches the processes, checks if the CPU usage exceeds a certain threshold it will extract the pid
  • Once we have the pid, we have a certain default threshold pre-defined and enforce it via cpulimit -p $pid -l $default_threshold command
  • This works fine with normal day to day linux processes but our data engineering team notified, when they run a certain process using python script that basically does read/write ops from Apache solr collections , the process shows exactly same output as shown in the video

Letme know if you want the code for the cpu-monitor service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants