Modular metrics for Node.js. Collect, process and publish metrics, picking only the metrics that you need.
Click to expand
- Plugin-based and evented
- A plugin serves one of 4 roles:
- Collector: emits metrics (when "pinged")
- Processor: decorates or combines metrics
- Schedule: pings other plugins on an interval
- Publisher: publishes metrics
- Add custom metrics sourced from a function, counter or your own plugin
- Add tags (a.k.a. dimensions) to metrics
- Includes plugins for automatic tags like
instanceidon EC2 - Locally aggregate before publishing metrics, to save bandwidth
- Publish to CloudWatch, AppOptics, Logz.io Metrics, stdio or your own publisher.
To get started you'll need 3 things.
1. The telemetry module (this)
This module controls what and when to collect and publish metrics. It groups plugins into "tasks". One task might publish a memory metric with a name tag to CloudWatch every 5 minutes, while another task publishes the same metric with a name, instanceid and region tag to AppOptics every minute. If the desired tags and interval are the same for both publishers, you can use a single task.
Tasks are also useful to manage the lifetime of metrics. Say we have an application with long-running background jobs that we want metrics on. Each job can have its own telemetry task and add for example a job_id tag to its metrics.
2. At least one collector and publisher
Without a collector, there are no metrics. Without a publisher, the metrics don't go anywhere.
3. A schedule like @telemetry-js/schedule-simple.
Most collectors follow a pull-based model: they must be "pinged" before they emit metrics. A schedule does just that, typically pinging your plugins on a fixed interval.
This example collects Node.js memory metrics every 30 seconds and sends them to CloudWatch with a name tag.
const telemetry = require('@telemetry-js/telemetry')()
const mem = require('@telemetry-js/collector-nodejs-memory')
const simple = require('@telemetry-js/schedule-simple')
const tag = require('@telemetry-js/processor-tag')
const cloudwatch = require('@telemetry-js/publisher-cloudwatch')
telemetry.task()
.collect(mem)
.schedule(simple, { interval: '30s' })
.process(tag, { name: 'my-app' })
.publish(cloudwatch)
// Start tasks
await telemetry.start()This example collects metrics every 30 seconds, locally aggregates them, and sends summary metrics to CloudWatch and AppOptics every 5 minutes. Those summary metrics have a min, max, sum and count of recorded values, and are called "statistic sets" in CloudWatch.
const telemetry = require('@telemetry-js/telemetry')()
const fn = require('@telemetry-js/collector-function')
const counter = require('@telemetry-js/collector-counter')
const disk = require('@telemetry-js/collector-disk')
const simple = require('@telemetry-js/schedule-simple')
const summarize = require('@telemetry-js/processor-summarize')
const tag = require('@telemetry-js/processor-ec2-instance-id')
const cloudwatch = require('@telemetry-js/publisher-cloudwatch')
const appoptics = require('@telemetry-js/publisher-appoptics')
// Example of custom metric that takes value from a function
const rand = fn.sync('myapp.random.count', { unit: 'count' }, Math.random)
const errors = counter.delta('myapp.errors.delta')
telemetry.task()
.collect(rand)
.collect(errors)
.collect(disk, { metrics: ['*.percent'] })
.schedule(simple, { interval: '30s' })
.process(summarize, { window: '5m' })
.process(tag)
.publish(cloudwatch)
.publish(appoptics, { token: '***' })
await telemetry.start()
// Elsewhere in your app
errors.increment(1)Got you:
const appoptics = require('@telemetry-js/publisher-appoptics')
const single = require('@telemetry-js/metric').single
const publisher = appoptics({ token: '***' })
const metric = single('myapp.example.count', { unit: 'count', value: 2 })
publisher.publish(metric)
await publisher.flush()| Name | Description |
|---|---|
| disk | Free, available and total disk space |
| net | TCP, UDP, ICMP, IP metrics |
| sockstat | /proc/net/sockstat metrics |
| nodejs-gc | Node.js garbage collection duration |
| nodejs-memory | Node.js memory (RSS, heap, external) |
| nodejs-event-loop-duration | Node.js event loop duration |
| nodejs-event-loop-lag | Node.js event loop lag |
| osmem | Free, used and total memory |
| counter | A counter incremented by you |
| function | Collect metric value from a function |
| redis | Redis metrics |
| incidental | Record incidental values |
| stopwatch | Record durations |
| aws-lb | AWS LB node count |
| dmesg | Count certain kernel messages |
| Name | Description |
|---|---|
| simple | Collect metrics on a fixed interval |
| Name | Description |
|---|---|
| summarize | Locally summarize metrics within a time window |
| tag | Add your own tags |
| ecs-tags | Add common tags for ECS container |
| ec2-instance-id | Add instanceid tag, fetched from metadata |
| ec2-instance-tags | Copy all instance tags, fetched from EC2 API |
| ec2-instance-name | Copy only the name tag (if set) |
| ec2-instance-region | Add region tag, fetched from metadata |
| debug | Log metrics and task lifecycle events with debug |
| Name | Description |
|---|---|
| appoptics | AppOptics |
| cloudwatch | CloudWatch |
| logzio-metrics | Logz.io Metrics |
| ndjson | Write NDJSON to a stream |
| debug | Log metrics and task lifecycle events with debug |
Lowercase, namespaced by dots (e.g. myapp.foo.bar.count), prefixed with project (myapp.), suffixed with unit (.count).
Metric names from Telemetry plugins are prefixed with telemetry. Custom metrics of your app should be prefixed with your appname. If however, your custom metric is not app-specific, then you could instead use metrics tags to differentiate apps and/or runtime contexts. You might as well write a Telemetry plugin at that point. Do it!
Lowercase, only a-z. E.g. instanceid, name, project, environment.
Follow the format <role>-<name>, where:
<role>is one ofcollector,processor,publisher,schedule, or, if the plugin serves multiple roles, then simplyplugin<name>typically matches the metric name (for collectors), purpose (for processors) or publisher name. If the plugin is a collector that emits multiple metrics (e.g.disk.free.bytes,disk.total.bytes) then use the longest common prefix as<name>(e.g.disk).
Yet to document.
With npm do:
npm install @telemetry-js/telemetry
This project is kindly sponsored by Reason Cybersecurity Ltd.
MIT © Vincent Weevers