Add OpenMetrics Metricbeat module

[ChrsMark]

What does this PR do?

This PR adds OpenMetrics Metricbeat module as a light module on top of Prometheus collector metricset.

Why is it important?

OpenMetrics is an effort to standardize exposing metric data based on the Prometheus exposition format. This standalone module will provide an easy and out of the box way to collect metrics from endpoints that follow this format.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

1. One can use a nodexporter as an example Prometheus app to collect from. For instance this forked project should be enough: https://github.com/ChrsMark/dockprom. Download and docker-compose up. Then metrics should be exported at http://localhost:9100/metrics.
2. Configuration:

- module: openmetrics
  metricsets: ['collector']
  period: 10s
  hosts: ['localhost:9100']

  # This module uses the Prometheus collector metricset, all
  # the options for this metricset are also available here.
  metrics_path: /metrics
  metrics_filters:
    include: ["node_filesystem_*"]
    exclude: ["node_filesystem_device_*", "^node_filesystem_readonly$"]

And expect to see only events that match node_filesystem_* but also not node_filesystem_device_* and are not node_filesystem_readonly.
3. Try to remove ^node_filesystem_readonly$ from exclude filters and see that this metric is included in events. One can try different combinations.

Related issues

• Relates [Metricbeat] OpenMetrics Module #14982

[jsoriano]

I have added some thoughts, but some things are going to depend on how OpenMetrics is released at the end. Let me know what you think.

In any case I would be ok with merging this by now as an initial implementation.

[jsoriano]

Are we ok with using prometheus in openmetrics fields?

In my opinion if we would be building prometheus/openmentrics support from scratch it could make sense to use openmetrics terminology, but given that we are already using prometheus, and most current implementations are prometheus-based, I am ok with using prometheus also for open metrics field names.

[ChrsMark]

I've no strong opinion on this. Actually it was sth I had as an open question too. Maybe @exekias or @sorantis could help with this decision.

[exekias]

+1 on using openmetrics namespace, specially since we expect the protocol & output to change a bit

[ChrsMark]

Tried to make of Namespace functionality that we already have so as to put metrics under openmetrics namespace. One implication is that this is gonna change the way we write event.dataset fields for prometheus bases modules because of this

beats/metricbeat/mb/event.go

Line 140 in fea2db3

"dataset": event.Namespace,

, here is a diff:

     "event": {
-        "dataset": "prometheus.collector",
+        "dataset": "prometheus",
         "duration": 115000,
         "module": "prometheus"
     },

@jsoriano @exekias would this be acceptable? I think this only affects prometheus, cochroachdb and ibmmq modules. wdyt?

[jsoriano]

@jsoriano @exekias would this be acceptable? I think this only affects prometheus, cochroachdb and ibmmq modules. wdyt?

I guess we should use prometheus.collector as namespace then? Or maybe we should review how the namespace affects to the value of event.dataset.

[ChrsMark]

@jsoriano @exekias would this be acceptable? I think this only affects prometheus, cochroachdb and ibmmq modules. wdyt?

I guess we should use prometheus.collector as namespace then? Or maybe we should review how the namespace affects to the value of event.dataset.

If we use prometheus.collector as namespace then we break the schema, since metrics would go under prometheus.collector.*.

[ChrsMark]

I see that this field was introduced at #8941.

In my view it is quite robust. We report prometheus metrics under prometheus.* so dataset makes sense to be prometheus too and not prometheus.collector and the same goes for light modules that are build on top of it since they report metrics under the same namespace.

So I would consider it safe to change it since there are still event.module and metricset.name which can be also used.

[jsoriano]

Oh wait, I don't like this change, I didn't fully get the implications of the namespace changes in prometheus collector.

It was intentional to have the name of the metricset in event.dataset. This way we have all the prometheus metrics under prometheus.metrics, but we can still use event.dataset to know the source of the metrics.

[jsoriano]

So I would consider it safe to change it since there are still event.module and metricset.name which can be also used.

Umm, this could be a breaking change if something or someone is using event.dataset for this purpouse (I would have used this field myself, and I have probably recommended it in the past).

[ChrsMark]

hmm, I got your point @jsoriano. A workaround could be what I tried and implemented, to pass the "namespace" to the factory method and use within the metricset accordingly. Let me know what you think.

ps: sorry for the force-push but I had to drop the commit that changed golden files.

[jsoriano]

LGTM, "too many open files" errors in CI are fixed in master.

[ChrsMark]

Merging this, since Libbeat test's(https://travis-ci.org/elastic/beats/jobs/658099652?utm_medium=notification&utm_source=github_status) are also failing on master.