Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exec driver CPU limit isn't honored when using cgroupv2 #11120

Closed
valodzka opened this issue Sep 1, 2021 · 2 comments · Fixed by #11287
Closed

exec driver CPU limit isn't honored when using cgroupv2 #11120

valodzka opened this issue Sep 1, 2021 · 2 comments · Fixed by #11287
Labels
stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/driver/exec type/bug

Comments

@valodzka
Copy link
Contributor

valodzka commented Sep 1, 2021

Nomad version

Nomad v1.1.4 (acd3d78)

Operating system and Environment details

Debian GNU/Linux 11 with cgroups v2

Issue

CPU limit ignored when using driver=exec

Reproduction steps

Run tasks with different CPU resources and exec driver. See example job attached.

Expected Result

Tasks with a higher cpu limit get more cpu.

Actual Result

To difference in cpu usage, see screenshot (stress -v and stress -q have same cpu usage):

image

Compared with driver="docker". stress with -v gets much lower cpu percent:

image

Job file (if appropriate)


 group "stress" {
   count = 1
   task "stress" {

     driver = "exec"

     config {
       command = "stress"
       args =  ["--cpu", "12", "-q"]
     }
   #  driver = "docker"
   #  config {
   #    image = "progrium/stress"
   #    args = ["--cpu", "12", "-q"]
   #  }

     resources {
       memory = 10
       cpu = 1000
     }
   }
 }

 group "stress2" {
   count = 1
   task "stress" {
     driver = "exec"

    config {
       command = "stress"
       args =  ["--cpu", "12", "-v"]
     }
   #   driver = "docker"
   #  config {
   #    image = "progrium/stress"
   #    args = ["--cpu", "12", "-v"]
   #  }

     resources {
       memory = 10
       cpu = 10
     }
   }
 }
@lgfa29 lgfa29 added stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/driver/exec labels Sep 2, 2021
@lgfa29 lgfa29 changed the title exec driver CPU limit isn't honored exec driver CPU limit isn't honored when using cgroups v2 Sep 2, 2021
@lgfa29 lgfa29 changed the title exec driver CPU limit isn't honored when using cgroups v2 exec driver CPU limit isn't honored when using cgroupsv2 Sep 2, 2021
@lgfa29
Copy link
Contributor

lgfa29 commented Sep 2, 2021

Hi @valodzka 👋

Thanks for the report. I was able to reproduce it when using cgroups v2. Using v1 it works, so it seems like we will need to update the task driver

@lgfa29 lgfa29 changed the title exec driver CPU limit isn't honored when using cgroupsv2 exec driver CPU limit isn't honored when using cgrousv2 Sep 2, 2021
@lgfa29 lgfa29 changed the title exec driver CPU limit isn't honored when using cgrousv2 exec driver CPU limit isn't honored when using cgroupv2 Sep 2, 2021
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/driver/exec type/bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants