Description

Coda Hale's Metrics package makes it easy to create useful metrics so you know what is going on in production. In addition to showing up in the normal Java places (JMX), Metrics supports an arbitrary number of Reporters (where to send the application telemetry to make pretty graphs). Ganglia and Graphite (or both!) are among the most popular choices.

However, Metrics purposefully doesn't come with a kitchen sink of support for every dependency injection or configuration tool yet devised by Java developers. Metrics-Reporter-Config aims to provide a simple way to configure and enable a set of Reporters that can be shared among applications. It should fit most (say 90% of) use cases and avoid situations like a plethora of subtly incompatible properties files.

In particular, it should seamlessly fit into Dropwizard service Configuration classes.

Use

Maven

<dependency>
  <groupId>com.addthis.metrics</groupId>
  <artifactId>reporter-config</artifactId>
  <version>2.0.5</version>
</dependency>

Basics

Current supported reporters include:

• console
• csv
• ganglia
• graphite
• influx
• prometheus

Note that Metrics-Reporter-Config's dependencies on non-core reporters are optional in maven-speak. That means that depending on Metrics-Reporter-Config should not pull in "a bunch of junk", but your applications will need to explicitly depend on the reporters they want (such as metrics-ganglia).

An example config file might look like this:

csv:
  -
    outdir: '/tmp/metrics/crazy-debugging'
    period: 2
    timeunit: 'MINUTES'
ganglia:
  -
    period: 60
    timeunit: 'SECONDS'
    hosts:
      - host: 'gmond.domain.local'
        port: 8649
      - host: 'gmond-backup.domain.local'
        port: 8649
graphite:
  -
    period: 120
    timeunit: 'SECONDS'
    hosts:
     - host: 'graphite-server.domain.local'
       port: 2003

And then to wire up your app call ReporterConfig.loadFromFileAndValidate() (or loadFromFile() if you aren't into validating), and then enableAll().

In addition to an inline list, the ganglia config supports gmondConf field which tells it where to find gmond's config file (ie /etc/ganglia/gmond.conf) and read any UDP send channels there. This should make it easy to keep the java app config and the rest of the system in sync.

Dropwizard

Just add a ReporterConfig instance to your app's Configuration class. Metrics-Reporter-Config uses the same validation implementation as dropwizard. You will still need to call an enable method when initialization your app.

Predicates

predicate:
  color: "black"
  patterns:
    - ".*JMXONLY$"

Each reporter can have a simple name based predicate with a white or black list of regular expressions. Use this for rarely used metrics that will only show up in JMX without flooding ganglia with 20k extra metrics. In practice our configurations tend to look like this:

ganglia:
  -
    period: 60
    timeunit: 'SECONDS'
    gmondConf: '/etc/ganglia/gmond.conf'
    predicate:
      color: "black"
      patterns:
      - ".*JMXONLY$"

By default predicates operate on the "Metric Name" (usually something like MyClass). If you want to operable on the conventional "full" name (org.example.foo.MyClass) enable useQualifiedName.

[EXPERIMENTAL] If you want to report only a subset of the measurements that are reported by a meter then use can use the measurement options on the predicate configuration. This feature is only available if you include as a dependency a fork of the metrics project that supports this feature. Forks supporting this feature: for metrics2: http://github.com/mspiegel/metrics; for metrics3 (graphite reporter only): http://github.com/thelastpickle/metrics. If the per measurement filtering is available then it is only applied once a metric has passed the top level filter.

ganglia:
  -
    period: 60
    timeunit: 'SECONDS'
    gmondConf: '/etc/ganglia/gmond.conf'
    predicate:
      color: "black"
      patterns:
      - ".*JMXONLY$"
      histogram:
          color: "white"
          patterns:
          - metric: ".*WidgetRunner$"
            measure: "mean|stddev"
          - metric: ".*SprocketRunner$"
            measure: ".*percentile"

Graphite and Ganglia

The Graphite and Ganglia reporters supports the "prefix" parameter. Its value will be prepended to each metric name sent by reporter. This feature is only available if you include as a dependency a fork of the metrics project that supports this feature. Forks supporting this feature: for metrics2: http://github.com/mspiegel/metrics; None yet known for metrics3. This value may contain one of the following variable references using "${variable.name}" as format.

- host.name = the value returned for local host by `java.net.InetAddress#getHostAddress()`
- host.name.short = same as ${host.name} up to first dot
- host.address = the value returned for local host by `java.net.InetAddress#getHostName()`
- host.fqdn = the value returned for local host by `java.net.InetAddress#getCanonicalHostName()`

Each substituted value gets sanitized by replacing all characters that are not allowed in the host name plus "-" with the underscore. The Ganglia reporter additionally supports the "groupPrefix" parameter. This will add a prefix to the Ganglia metric group.

Zabbix

The Zabbix reporter requires the dependency zabbix-sender-0.0.3.jar (Maven group ID: io.github.hengyunabc, Maven artifact ID: zabbix-sender, version: 0.0.3).

InfluxDB

The InfluxDB reporter currently requires the dependency metrics-influxdb-1.1.7.jar Sources (Maven group ID: com.izettle, Maven artifact ID: metrics-influxdb, version 1.1.7).

Metrics 4.0 will introduce native InfluxDB support.

InfluxDB connection requires a few more configuration properties:

influxdb:
  -
    period: 60
    timeunit: 'SECONDS'
    protocol: 'http'
    auth: 'troll:admin123'
    dbName: 'db1'
    connectionTimeout: 1000
    readTimeout: 1000
    tags:
     host: '${host.name}'
     env: 'prod'
    measurementMappings:
     cassandra_Table_ReadLatency: org.apache.cassandra.metrics.Table.ReadLatency.*
    prefix: ''
    hosts:
      - host: 'localhost'
        port: 8086

• hosts - must contain exactly one host configured. For HA see the InfluxDB Relay
• protocol - http or https
• auth - username:password, can be empty
• dbName - InfluxDB database name
• connectionTimeout - java.net.URLConnection connection timeout in milliseconds
• readTimeout - java.net.URLConnection read timeout in milliseconds
• tags - every send measurement will be tagged with this tags map. Tags values can contain variables described for Graphite prefix parameter - host.name, host.name.short, host.address, host.fqdn.
• prefix - its value will be prepended to each metric name sent by reporter (see Graphite & Ganglia). Prefix can contain variables described for Graphite prefix parameter - host.name, host.name.short, host.address, host.fqdn. Please also look at tags.

Prometheus

only metric3 is currently supported

The Prometheus configuration provides two options to report metrics. The first is the push gateway and the second is the metrics servlet.

• The push gateway has a dependency on io.prometheus:simpleclient_pushgateway:0.0.16
• The metrics service has a dependency on io.prometheus:simpleclient_servlet:0.0.16

Prometheus Specific Configuration Options:

• job - prometheus job name. This is added to every metric
• labels - map<string, string> that will be added to every metric. Variable substitution is available for label values (see graphite & ganglia)
• type - (pushgateway|servlet) pushgateway requires host(s), period and timeunit. Servlet does not.

Push Gateway Configuration

prometheus:
  -
    type: pushgateway
    period: 60
    timeunit: 'SECONDS'
    job: 'app_name'
    labels:
        env: test
        host: '${host.name}'
    hosts:
        - host: localhost
          port: 9091

Servlet Configuration

prometheus:
  -
    type: servlet
    job: 'app_name'
    labels:
        env: test
        host: '${host.name}'

In addition to the yaml configuration, you will also need to register the servlet with your environment.

config.getReporterConfig().enablePrometheus(environment.metrics());
MetricsServlet servlet = config.getReporterConfig().getPrometheus().get(0).getMetricsServlet();
environment.admin()
           .addServlet("prometheus-metrics", servlet)
           .addMapping("/prometheus-metrics");

Riemann

In order to use the riemann reporter, you need the following dependencies:

• riemann-java-client-0.2.10.jar (Maven group ID: com.aphyr, Maven artifact ID: java-client, version: 0.2.10).
• netty-all-3.6.1.Final.jar (Maven group ID: io.netty, Maven artifact ID: netty, version: 3.6.1.Final)
• protobuf-java-2.5.0.jar (Maven group ID: com.google.protobuf, Maven artifact ID: protobuf-java, version: 2.5.0)

Building

Assuming you have Apache Maven installed and configured:

mvn package

And you should be all set.

To run individual tests use -Dtest= and -Dorg.slf4j.simpleLogger.defaultLogLevel= to change the test log level.

Administrivia

Versioning

It's x.y.z where:

• x: compatible Metrics major version
• y: next release
• z: bug fix only

License

Metrics-Reporter-Config is released under the Apache License Version 2.0. See Apache or the LICENSE file in this distribution for details.

Where People Hang Out

Main Metrics mailing list: http://groups.google.com/group/metrics-user