-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker containers use swap in 0.9.4 #6085
Comments
Thanks for this! We'll need to do some more investigation and document this behavior or change it back. As a workaround, docker inherits
|
Indeed, we had to disable swap globally. |
@pznamensky did you happen to see this? https://news.ycombinator.com/item?id=20651510 The article references a thread from this week on the linux kernel mailing list, the summary offered in the article is that leaving at least a bit of swap enabled on even the latest linux kernel allows the system to avoid some very ugly behavior under memory pressure. You may want to investigate leaving the global swap enabled and running nomad in a cgroup instead. Cheers |
@langmartin sorry for the delay and thanks for the link. Indeed, this is an old issue with swap, and we make a decision to use a little swap in our production, but I expected that nomad disables it for containers as described here: |
Just reached here after @honnibal pointed it out. I would love to leave the swappiness behavior as-is. I wonder how Nomad 0.9.5 and 0.10-beta1 behave. I am for Nomad allowing "swap". 🦊 |
zram linux module use swap mechanism for compressing memory in desktop/server/etc. |
1.) Please correct me if I'm wrong. It seems to me that the nomad documentation is wrong. https://www.nomadproject.io/docs/drivers/docker.html#memory states:
But docker containers scheduled via nomad are allowed to use swap: Nomad should set 2.) It would be great to be able to configure the |
Is there a plan to fix it? |
MemorySwappiness can only be set in non-Windows options: https://ci.appveyor.com/project/hashicorp/nomad/builds/31832149 Also fixes #6085
I am confused with the various issues around this. What is the latest news with versions 0.10.x and the 0.11.x? |
@shantanugadgil The PR #7550 does indeed disable swap, as we never intended to support it. We can entertain adding a flag or supporting swappiness if there is a demand. Mind if you elaborate on your case? Do you currently rely on current swap behavior and how so? Would you like more swap knobs at the client/job levels. If I understand correctly, nomad scheduling only factor in available RAM on the host, so swap memory should only be used if non-nomad work is using up memory and/or some of the nomad tasks are unconstrained (e.g. raw_exec) and use more memory than declared. Does that seem accurate to you? |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad version
Nomad v0.9.4 (a81aa84)
Operating system and Environment details
CentOS Linux release 7.6.1810 (Core)
Docker version 18.03.1-ce, build 9ee9f40
Issue
After updating nomad from 0.8.4 to 0.9.4 docker containers started to use swap.
Despite this:
https://github.com/hashicorp/nomad/blob/v0.9.4/drivers/docker/driver.go#L730
docker inspect
shows that containers use unlimited swap:Reproduction steps
MemorySwap
parameter of the containerThe text was updated successfully, but these errors were encountered: