Usage

This page contains information regarding the usage of l2met. Usage includes how to drain an apps logs into l2met and what sort of data can be extracted from the logs. If you are interested in how l2met works, checkout the architecture page. If you are interested in the system administration of l2met, checkout the administration page.

L2met has two basic inputs: the drain URL and the log line. The log lines are delivered to the drain URL over HTTPs. Thus, there are 3 topics related to l2met usage:

• Logging Convention
• Drain URLS
• Log Delivery

Logging Convention

Logging is defined to be a stream of bytes emitted by a process to STDOUT. Ultimately, l2met expects the log lines to be delivered in RFC5424 frames, but understanding framing is not necessary in understanding the logging convention. More details about framing can be found in the section on Log Delivery. Thus, the following ruby snippet is an example of a log:

$stdout.puts("hello world")

As an application developer, this is your interface into l2met. Simply write a string to STDOUT, and you will be producing metrics in no time. Here is an example of a ruby process emitting a counter metric:

$stdout.puts("count#bottles-of-beer-on-the-wall=1")

In the string printed to STDOUT, you will notice a key=value pair. The key contains a metric type, followed by a # followed by the metric name. The value simply contains a number. Optionally you can attach a unit to the number. E.g.

$stdout.puts("count#beers-on-the-wall=1bottle")

There are 3 metrics types: counters, samples, and measurements.

# A counter.
$stdout.puts("count#user.clicks=4")
# A Sample.
$stdout.puts("sample#database.size=40.9MB")
# A Measurement.
$stdout.puts("measure#database.query=200ms")

Counters will emit the sum of the collected values. Samples will emit the last value seen. Measurements will emit the min, median, 95th & 99th percentile, max, mean, sum, and count.

You can combine multiple metrics into a single line:

$stdout.puts("count#signups=1 measure#signup-controller=50ms sample#queue.length=99")

The value of metrics will parsed into a 64bit floating point number. The unit must immediately follow the number and must only include a-zA-Z.

Finally, there is one last keyword to address. The source keyword. This is handy for distinguishing machines or environments. You can associate a set of metrics with a source by adding a source key=value pair to the log line. E.g.

$stdout.puts("source=us-east measure#web.latency=4ms")

The Heroku Router

Heroku's router does not follow the l2met logging convention. However, l2met has a builtin parser that will compute the following metrics on the router's logs. Basically, the router log lines are transformed into:

"measure#service=4ms measure#connect=2ms measure#bytes=5"

Drain URLs

Log lines are delivered to l2met via HTTPs. Thus, you will need a URL in order to send your log stream to l2met. The address url of your l2met server is known as the Drain URL. In addition to holding the host of the l2met server, the Drain URL also contains:

• Your Librato API credentials. (Signed and encrypted)
• The resolution at which metrics sent to the URL are to be aggregated. (e.g. 1s 5s 10s 60s)
• A source prefix which will be prepended to all source keys on your logs lines.
• A name prefix which will be prepended to all metric names on your log lines.

L2met does not permanently store any Librato credentials. Therefore, when setting up your Drain URL, you will need to include your encrypted credentials in the user portion of the URL. You can use l2met's sign api to encrypt the Librato credentials. To do so, you will need access to the $SECRETS variable which is set on the l2met server. Here is an example curl(1) command to sign a set of Librato credentials:

$ export libratouser=r@32k.io
$ export libratopass=abc123
$ export secret=def456 # must be set as the $SECRETS env var on your l2met server
$ curl -s "https://my-l2met.net/sign" --data "$libratouser:$libratopass" -u "$secret:"
gAAAAABR-UAfqpquaZLK77ROMwL3dHXtuHC8jzTVRc6LLsPIPn9Vbe5yVqZ2aRjfyt82KH7mn9iEkah8tO4udeyDH8bguepTebR_tOqnO0bwQuCyIjG-hABav8NmxbrKPGsq5FvdL3ZjdyHr2ewjEWQRJIEKs8vqNlUrBRQKSLqxbtBHiVmWwmM=

The value that is returned from the /sign api is your signed & encrypted Librato credentials. You place them in the user portion of the Drain URL. E.g.

'https://gAAAAABR-UAfqpquaZLK77ROMwL3dHXtuHC8jzTVRc6LLsPIPn9Vbe5yVqZ2aRjfyt82KH7mn9iEkah8tO4udeyDH8bguepTebR_tOqnO0bwQuCyIjG-hABav8NmxbrKPGsq5FvdL3ZjdyHr2ewjEWQRJIEKs8vqNlUrBRQKSLqxbtBHiVmWwmM=@my-l2met.net/logs'