Skip to content

Commit

Permalink
docker_logger: reorder imports to save memory (#14875)
Browse files Browse the repository at this point in the history
Nomad runs one logmon process and also one docker_logger process for each
running allocation. A naive look at memory usage shows 10-30 MB of RSS, but a
closer look shows that most of this memory (ex. all but ~2MB for logmon) is
shared (`Shared_Clean` in Linux pmap).

But a heap dump of docker_logger shows that it currently has an extra ~2500 KiB
of heap (anonymously-mapped unshared memory) used for init blocks coming from
the agent code (ex. mostly regexes from go-version, structs, and the Consul
SDK). The packages for running logmon, docker_logger, and executor have an init
block that parses `os.Args` to drop into their own logic, which prevents them
from loading all the rest of the agent code and saves on memory, so this was
unexpected.

It looks like we accidentally reordered the imports in main to undo some of the
work originally done in 404d2d4. This changeset
restores the ordering. A follow-up heap dump shows this saves ~2MB of unshared
RSS per docker_logger process.
  • Loading branch information
tgross committed Oct 11, 2022
1 parent f91100b commit b3baaa2
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
3 changes: 3 additions & 0 deletions .changelog/14875.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:improvement
docker: improve memory usage for docker_logger
```
6 changes: 4 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ import (
// These packages have init() funcs which check os.Args and drop directly
// into their command logic. This is because they are run as separate
// processes along side of a task. By early importing them we can avoid
// additional code being imported and thus reserving memory
// additional code being imported and thus reserving memory.
_ "github.com/hashicorp/nomad/client/logmon"
"github.com/hashicorp/nomad/command"
_ "github.com/hashicorp/nomad/drivers/docker/docklog"
_ "github.com/hashicorp/nomad/drivers/shared/executor"

// Don't move any other code imports above the import block above!
"github.com/hashicorp/nomad/command"
"github.com/hashicorp/nomad/version"
"github.com/mitchellh/cli"
"github.com/sean-/seed"
Expand Down

0 comments on commit b3baaa2

Please sign in to comment.