Adds statsd/graphite Based Telemetry for Nomad

[Miserlou]

Issue Number

#54

Purpose/Implementation Notes

This adds statsd+graphite based telemetry to Nomad. This will be necessary for understanding the cpu/disk/memory resource usages of active systems so that we can further refine our usage job specifications.

Types of changes

• New feature (non-breaking change which adds functionality)

Functional tests

Not applicable, but tested locally.

Can be tested locally by running:

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 graphiteapp/graphite-statsd

then adding this to local nomad config:

telemetry {
  publish_allocation_metrics = true
  publish_node_metrics       = true
  statsd_address = "localhost:8125"
}

and visiting localhost:80 in the browser.

Checklist

Put an x in the boxes that apply.

• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

Screenshots

[kurtwheeler]

This looks good to me. I'm curious if it's possible for the lead server to publish stats on itself as well or is that a chicken and egg type problem?