[Metricbeat][Istio] Add pilot metricset

metricbeat/docs/fields.asciidoc

@@ -16901,6 +16901,203 @@ type: keyword
 
 --
 
+[float]
+=== pilot
+
+Contains statistics related to the Istio pilot service
+
+
+
+*`istio.pilot.xds.count`*::
++
+--
+Count of concurrent xDS client connections for Pilot.
+
+
+type: integer
+
+--
+
+*`istio.pilot.xds.pushes`*::
++
+--
+Count of xDS messages sent, as well as errors building or sending xDS messages for lds, rds, cds and eds.
+
+
+type: integer
+
+--
+
+*`istio.pilot.xds.push.time.sec`*::
++
+--
+Total time in seconds Pilot takes to push lds, rds, cds and eds.
+
+
+type: long
+
+--
+
+*`istio.pilot.xds.eds.instances`*::
++
+--
+Instances for each cluster, as of last push. Zero instances is an error.
+
+
+type: object
+
+--
+
+*`istio.pilot.xds.push.context.errors`*::
++
+--
+Number of errors (timeouts) initiating push context.
+
+
+type: integer
+
+--
+
+*`istio.pilot.xds.internal.errors`*::
++
+--
+Total number of internal XDS errors in pilot.
+
+
+type: integer
+
+--
+
+*`istio.pilot.conflict.inbound.listener`*::
++
+--
+Number of conflicting inbound listeners.
+
+
+type: integer
+
+--
+
+*`istio.pilot.conflict.outbound.listener.http.over.current.tcp`*::
++
+--
+Number of conflicting wildcard http listeners with current wildcard tcp listener.
+
+
+type: integer
+
+--
+
+*`istio.pilot.conflict.outbound.listener.http.over.https`*::
++
+--
+Number of conflicting HTTP listeners with well known HTTPS ports.
+
+
+type: integer
+
+--
+
+*`istio.pilot.conflict.outbound.listener.tcp.over.current.http`*::
++
+--
+Number of conflicting wildcard tcp listeners with current wildcard http listener.
+
+
+type: integer
+
+--
+
+*`istio.pilot.conflict.outbound.listener.tcp.over.current.tcp`*::
++
+--
+Number of conflicting tcp listeners with current tcp listener.
+
+
+type: integer
+
+--
+
+*`istio.pilot.proxy.conv.ms.bucket.*`*::
++
+--
+Time needed by Pilot to push Envoy configurations, histogram buckets in milliseconds.
+
+
+type: object
+
+--
+
+*`istio.pilot.proxy.conv.ms.sum`*::
++
+--
+Time needed by Pilot to push Envoy configurations, histogram sum of times in milliseconds.
+
+
+type: long
+
+--
+
+*`istio.pilot.services`*::
++
+--
+Total services known to pilot.
+
+
+type: integer
+
+--
+
+*`istio.pilot.virt.services`*::
++
+--
+Total virtual services known to pilot.
+
+
+type: integer
+
+--
+
+*`istio.pilot.proxy.conv.ms.count`*::
++
+--
+Time needed by Pilot to push Envoy configurations, histogram count of times.
+
+
+type: long
+
+--
+
+*`istio.pilot.no.ip`*::
++
+--
+Pods not found in the endpoint table, possibly invalid.
+
+
+type: object
+
+--
+
+*`istio.pilot.cluster`*::
++
+--
+The instance FQDN.
+
+
+type: text
+
+--
+
+*`istio.pilot.type`*::
++
+--
+The Envoy proxy configuration type.
+
+
+type: text
+
+--
+
 [[exported-fields-jolokia]]
 == Jolokia fields
 

metricbeat/docs/modules/istio.asciidoc

@@ -11,7 +11,7 @@ beta[]
 This is the Istio module. The Istio module collects metrics from the
 Istio https://istio.io/docs/tasks/observability/metrics/querying-metrics/#about-the-prometheus-add-on[prometheus exporters endpoints].
 
-The default metricset is `mesh`.
+The default metricsets are `mesh`, `pilot`.
 
 [float]
 === Compatibility
@@ -32,6 +32,13 @@ metricbeat.modules:
   metricsets: ["mesh"]
   period: 10s
   hosts: ["localhost:42422"]
+
+# Istio pilot. To collect all Pilot-generated metrics.
+- module: istio
+  metricsets: ["pilot"]
+  period: 10s
+  # use istio-pilot.istio-system:15014, when deploying Metricbeat in a kuberentes cluster
+  hosts: ["localhost:15014"]
 ----
 
 [float]
@@ -41,5 +48,9 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-istio-mesh,mesh>>
 
+* <<metricbeat-metricset-istio-pilot,pilot>>
+
 include::istio/mesh.asciidoc[]
 
+include::istio/pilot.asciidoc[]
+

metricbeat/docs/modules/istio/pilot.asciidoc

@@ -0,0 +1,23 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+[[metricbeat-metricset-istio-pilot]]
+=== istio pilot metricset
+
+beta[]
+
+include::../../../../x-pack/metricbeat/module/istio/pilot/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-istio,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../../x-pack/metricbeat/module/istio/pilot/_meta/data.json[]
+----

metricbeat/docs/modules_list.asciidoc

@@ -104,7 +104,8 @@ This file is generated! See scripts/mage/docs_collector.go
 |<<metricbeat-module-ibmmq,IBM MQ>>  beta[]   |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
 .1+| .1+|  |<<metricbeat-metricset-ibmmq-qmgr,qmgr>> beta[]  
 |<<metricbeat-module-istio,istio>>  beta[]   |image:./images/icon-no.png[No prebuilt dashboards]    |  
-.1+| .1+|  |<<metricbeat-metricset-istio-mesh,mesh>> beta[]  
+.2+| .2+|  |<<metricbeat-metricset-istio-mesh,mesh>> beta[]  
+|<<metricbeat-metricset-istio-pilot,pilot>> beta[]  
 |<<metricbeat-module-jolokia,Jolokia>>     |image:./images/icon-no.png[No prebuilt dashboards]    |  
 .1+| .1+|  |<<metricbeat-metricset-jolokia-jmx,jmx>>   
 |<<metricbeat-module-kafka,Kafka>>     |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  

x-pack/metricbeat/metricbeat.reference.yml

@@ -543,6 +543,13 @@ metricbeat.modules:
   period: 10s
   hosts: ["localhost:42422"]
 
+# Istio pilot. To collect all Pilot-generated metrics.
+- module: istio
+  metricsets: ["pilot"]
+  period: 10s
+  # use istio-pilot.istio-system:15014, when deploying Metricbeat in a kuberentes cluster
+  hosts: ["localhost:15014"]
+
 #------------------------------- Jolokia Module -------------------------------
 - module: jolokia
   #metricsets: ["jmx"]

x-pack/metricbeat/module/istio/_meta/config.reference.yml

@@ -2,3 +2,10 @@
   metricsets: ["mesh"]
   period: 10s
   hosts: ["localhost:42422"]
+
+# Istio pilot. To collect all Pilot-generated metrics.
+- module: istio
+  metricsets: ["pilot"]
+  period: 10s
+  # use istio-pilot.istio-system:15014, when deploying Metricbeat in a kuberentes cluster
+  hosts: ["localhost:15014"]

x-pack/metricbeat/module/istio/_meta/config.yml

@@ -2,3 +2,10 @@
   metricsets: ["mesh"]
   period: 10s
   hosts: ["localhost:42422"]
+
+# Istio pilot. To collect all Pilot-generated metrics.
+- module: istio
+  metricsets: ["pilot"]
+  period: 10s
+  # use istio-pilot.istio-system:15014, when deploying Metricbeat in a kuberentes cluster
+  hosts: ["localhost:15014"]

x-pack/metricbeat/module/istio/_meta/docs.asciidoc

@@ -1,7 +1,7 @@
 This is the Istio module. The Istio module collects metrics from the
 Istio https://istio.io/docs/tasks/observability/metrics/querying-metrics/#about-the-prometheus-add-on[prometheus exporters endpoints].
 
-The default metricset is `mesh`.
+The default metricsets are `mesh`, `pilot`.
 
 [float]
 === Compatibility

x-pack/metricbeat/module/istio/pilot/_meta/data.json

@@ -0,0 +1,26 @@
+{
+    "@timestamp": "2019-03-01T08:05:34.853Z",
+    "event": {
+        "dataset": "istio.pilot",
+        "duration": 115000,
+        "module": "istio"
+    },
+    "istio": {
+        "pilot": {
+            "cluster": "outbound_.15031_._.istio-ingressgateway.istio-system.svc.cluster.local",
+            "xds": {
+                "eds": {
+                    "instances": 1
+                }
+            }
+        }
+    },
+    "metricset": {
+        "name": "pilot",
+        "period": 10000
+    },
+    "service": {
+        "address": "127.0.0.1:55555",
+        "type": "istio"
+    }
+}

x-pack/metricbeat/module/istio/pilot/_meta/docs.asciidoc

@@ -0,0 +1 @@
+This is the pilot metricset of the module istio. This metricset collects all Pilot-generated metrics.