-
Notifications
You must be signed in to change notification settings - Fork 34
/
data-sources.html.md.erb
272 lines (220 loc) · 9.25 KB
/
data-sources.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
---
title: Logging and metrics in Cloud Foundry
owner: Logging and Metrics
---
You can use logging and metrics in <%= vars.app_runtime_full %>
(<%= vars.app_runtime_abbr %>). Learn about logs and metrics sources, and
transport systems. It also lists products for viewing logs and metrics.
## <a id="sources"></a> Accessing metrics
You must use a Cloud Foundry Command-Line Interface (cf CLI) plug-in to access and view metrics
directly from the command line. You can use either the Firehose plug-in or the Log Cache plug-in.
### <a id='log-cache'></a> Accessing metrics using the Log Cache CLI plug-in
To access metrics with the Log Cache plug-in:
1. Log in to the cf CLI by running:
```
cf login -a API-URL -u USERNAME -p PASSWORD
```
Where:
<ul>
<li><code>API-URL</code> is your API endpoint.</li>
<li><code>USERNAME</code> is your username.</li>
<li><code>PASSWORD</code> is your password.</li>
</ul>
1. Install the Log Cache CLI plug-in by running:
```
cf install-plug-in -r CF-Community "log-cache"
```
For more information, see [log-cache](https://plugins.cloudfoundry.org/#log-cache) in _cf
CLI plug-ins_ on the Cloud Foundry website.
1. Run:
```
cf query 'METRIC-NAME{source_id="SOURCE-ID"}'
```
Where:
<ul>
<li><code>METRIC-NAME</code> is the name of the metric you want to view.</li>
<li><code>SOURCE-ID</code> is the source ID of the component for which you want to view metrics.</li>
</ul>
To find the source ID and metric name of the metric you want to view, <%= vars.find_metric_name_source_id %>.
### <a id='cf-nozzle'></a> Accessing metrics using the Firehose plug-in
To access metrics using the Firehose plug-in:
1. Log in to the cf CLI by running:
```
cf login -a API-URL -u USERNAME -p PASSWORD
```
Where:
<ul>
<li><code>API-URL</code> is your API endpoint.</li>
<li><code>USERNAME</code> is your username.</li>
<li><code>PASSWORD</code> is your password.</li>
</ul>
1. Install the Firehose cf CLI plug-in by running:
```
cf install-plugin -r CF-Community "Firehose Plugin"
```
For more information, see [Firehose plug-ins](https://plugins.cloudfoundry.org/#Firehose%20Plugin)
in _cf CLI Plug-ins_ on the Cloud Foundry website.
1. Run:
```
cf nozzle -no-filter | grep SOURCE-ID | grep -i METRIC-NAME
```
Where:
<ul>
<li><code>METRIC-NAME</code> is the name of the metric you want to view.</li>
<li><code>SOURCE-ID</code> is the source ID of the component for which you want to view metrics.</li>
</ul>
For example:
```
cf nozzle -no-filter | grep bbs | grep -i ConvergenceLRPDuration
```
Because metrics are scraped at different intervals, it might take up to fifteen minutes for
the Firehose to receive all metrics for the component. Run `cf nozzle` again until you have
received a complete set of metrics for a component. For more information, see [Firehose
Architecture](../loggregator/architecture.html#firehose-architecture) in _Loggregator Architecture_.
<br>
<br>
To find the source ID and metric name of the metric you want to view, <%= vars.find_metric_name_source_id %>.
For more information about nozzles, <%= vars.nozzle_info_link %>.
## <a id="sources"></a> Sources for logs and metrics
The following sources are for <%= vars.app_runtime_abbr %> logs and metrics:
* <%= vars.app_runtime_abbr %> platform components, such as a Diego Cell, MySQL Server, or
Cloud Controller
* Apps and app containers deployed on <%= vars.app_runtime_abbr %>
The following table describes the data included in logs and metrics from each source:
<table class=“table”>
<thead>
<tr>
<th width="20%">Source</th>
<th width="40%">Logs data</th>
<th>Metrics data</th>
</tr>
</thead>
<tr>
<td>Platform components</td>
<td>Logs from <%= vars.app_runtime_abbr %> components</td>
<td>
<ul>
<li>Health metrics from BOSH-deployed VMs<sup>1</sup></li>
<li>Platform metrics from <%= vars.app_runtime_abbr %> components. For example, Diego
Cell capacity remaining and Gorouter throughput.</li>
<li>Metrics for any service tile that self-publishes to the Loggregator Firehose. For
example, Redis and MySQL.</li>
</ul>
</td>
</tr>
<tr>
<td>Apps and app containers</td>
<td>Logs from apps<sup>2</sup></td>
<td>
<ul>
<li>Container metrics<sup>3</sup></li>
<% if vars.platform_code != 'CF' %>
<li>Custom app metrics<sup>4</sup></li>
<% end %>
</ul>
</td>
</tr>
</table>
<% if vars.platform_code == 'CF' || vars.platform_code == 'OFFLINE' %>
<sup>1</sup>For more information about using the BOSH Health Monitor to collect health metrics
on VMs, see [Selecting and configuring a monitoring system](https://docs.vmware.com/en/VMware-Tanzu-Application-Service/<%= vars.current_major_version %>/tas-for-vms/monitoring-metrics.html).
<sup>2</sup>For more information about app logging, see [App Logging in TAS for VMs](https://docs.vmware.com/en/VMware-Tanzu-Application-Service/<%= vars.current_major_version %>/tas-for-vms/deploy-apps-streaming-logs.html).
<% else %>
<sup>1</sup>For more information about using the BOSH Health Monitor to collect health metrics
on VMs, see [Configuring a Monitoring System](/operating/monitoring/metrics.html).
<sup>2</sup>For more information about app logging, see [App Logging in TAS for VMs](/devguide/deploy-apps/streaming-logs.html).
<% end %>
<sup>3</sup>For more information about container metrics, see [Container Metrics](container-metrics.html).
<%= vars.custom_metrics_footnote %>
## <a id="transport"></a> Transport Systems for logs and metrics
The following transport systems deliver logs and metrics from their source to an observability
product for viewing:
* **Loggregator:** Loggregator is the transport system for both logs and metrics on apps deployed
on <%= vars.app_runtime_abbr %>, as well as metrics on <%= vars.app_runtime_abbr %> platform
components.
For more information about the Loggregator system, including Loggregator architecture
and components, see [Loggregator Architecture](architecture.html) and for the types
of envelopes being transported in Loggregator, see [Envelope Types](https://github.com/cloudfoundry/loggregator-api#v2-envelope-types).
* **rsyslogd on <%= vars.app_runtime_abbr %> component VMs:** rsyslogd is the transport system
for <%= vars.app_runtime_abbr %> component logs. Users can configure rsyslogd to transport
component logs to a third-party syslog server.
The following table lists the transport system for logs and metrics on <%= vars.app_runtime_abbr %>
platform components and apps:
<table class=“table”>
<thead>
<tr>
<th width="20%">Source</th>
<th width="40%">Logs transport system</th>
<th>Metrics transport system</th>
</tr>
</thead>
<tr>
<td>Platform components</td>
<td>rsyslogd on <%= vars.app_runtime_abbr %> component VMs</td>
<td>Loggregator</td>
</tr>
<tr>
<td>Apps</td>
<td>Loggregator</td>
<td>Loggregator</td>
</tr>
</table>
## <a id="viewing"></a> Viewing logs and metrics
The following table lists the products and tools for viewing <%= vars.app_runtime_abbr %> logs
and metrics:
<table class=“table”>
<thead>
<tr>
<th width="20%">Source</th>
<th width="40%">Products and tools for viewing logs</th>
<th>Products and tools for viewing metrics</th>
</tr>
</thead>
<tr>
<td>Platform components</td>
<td valign="top">To view system logs from <%= vars.app_runtime_abbr %> components, configure
rsyslogd to transport logs to a third-party product.</td>
<td valign="top">
You can use the following products or tools to view platform component and VM metrics:
<ul>
<li>Loggregator Firehose CLI Plug in. See <a href="cli-plugin.html">Installing the Loggregator
Firehose plug-in for CLI</a>.</li>
<li>Loggregator Log Cache CLI plug-in. See <a href="https://plugins.cloudfoundry.org/">
Cloud Foundry Community cf CLI plug-ins</a>.</li>
<% if vars.platform_code != 'CF' %>
<li><%= vars.platform_metric_tool_ref %></li>
<% end %>
</ul>
</td>
</tr>
<tr>
<td>Apps</td>
<td valign="top">
You can use the following products or tools to view app logs:
<ul>
<li>cf CLI cf logs command. See <a href="https://cli.cloudfoundry.org/en-US/cf/logs.html">
Cloud Foundry CLI Reference Guide.</a></li>
<% if vars.platform_code != 'CF' %>
<li><%= vars.manage_apps_service_instances_ref %></li>
<% end %>
<li><%= vars.syslog_forwarding_ref %></li>
<li>Loggregator Firehose CLI plug-in. See <a href="cli-plugin.html">Installing the Loggregator
Firehose plug-in for CLI</a>.</li>
<li>Loggregator Log Cache CLI plug-in. See <a href="https://plugins.cloudfoundry.org/">
Cloud Foundry Community cf CLI plug-ins</a>.</li>
</ul>
</td>
<td valign="top">
You can use the following products or tools to view app metrics:
<ul>
<li>Loggregator Firehose CLI plug-in. See <a href="cli-plugin.html">Installing the Loggregator
Firehose plug-in for CLI</a>.</li>
<li>Loggregator Log Cache CLI plug-in. See <a href="https://plugins.cloudfoundry.org/">
Cloud Foundry Community cf CLI plug-ins</a>.</li>
<% if vars.platform_code != 'CF' %>
<li><%= vars.metrics_product_ref %></li>
<% end %>
</ul>
</td>
</tr>
</table>