forked from pivotal-cf/docs-splunk
-
Notifications
You must be signed in to change notification settings - Fork 1
/
installing.html.md.erb
123 lines (104 loc) · 8.57 KB
/
installing.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
---
title: Installing and Configuring Splunk Firehose Nozzle for VMware Tanzu
owner: Partners
---
This topic describes how to install and configure Splunk Firehose Nozzle for VMware Tanzu.
##<a id='install'></a> Install and Configure Splunk Firehose Nozzle for VMware Tanzu
###Prerequisites
+ For information about the Ops Manager Installation Dashboard, see [Using Ops Manager](http://docs.pivotal.io/pivotalcf/1-11/customizing/).
+ There is no upgrade path from the previous release of Splunk Nozzle for VMware Tanzu (Beta). Install this version to an environment without the Splunk Firehose Nozzle for VMware Tanzu (Beta).
###Steps
1. Download the Splunk Firehose Nozzle file from [Pivotal Network](https://network.pivotal.io/products/splunk-nozzle/).
1. Navigate to the Ops Manager Installation Dashboard and click **Import a Product** to upload the product file.
1. Click **Add** next to the uploaded Splunk Firehose Nozzle for VMware Tanzu tile in the Ops
Manager **Available Products** view to add it to your staging area.
1. Click the newly added **Splunk Firehose Nozzle for VMware Tanzu** tile.
1. [Configure the tile](#configure) as described below.
1. Click **Save**.
1. Return to the Ops Manager Installation Dashboard and click **Apply Changes**
to install Splunk Firehose Nozzle for VMware Tanzu tile.
##<a id='configure'></a> Configuration
1. Click **Assign AZs and Networks**: Choose placement accordingly.
1. Click Splunk Settings:
Complete the fields as follows:
* **HTTP Event Collector Endpoint (HEC) URL**: HTTP Event Collector endpoint URL.`<http(s)>://<address>:<port>`,
where `<address>` is either FQDN or IP address,
and `<port>` is the port on which Splunk HEC is listening.
In a clustered environment this can also be the address and port of a load balancer that distributes requests
to multiple configured HEC endpoints. These endpoints can be across both Splunk Heavy Forwarders or Splunk Indexers.
* **HTTP Event Collector (HEC) Token**: Splunk generated HEC token.
[More Information](http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector)
* **Skip SSL validation**: Skip SSL certificate validation for connections to Splunk.
Secure communications will not check SSL certificates against a trusted Certificate Authority.
Do not use for a production environment.
* **Index**: The Splunk index that events are sent to.
<p class="note warning"><strong>WARNING</strong>: Setting an invalid index causes events to be lost.
This index must match one of the selected indexes for the Splunk HTTP event collector token used
for the HTTP Event Collector Token parameter.</p>
1. Click **Cloud Foundry Settings**:
Complete the fields as follows:
* **API Endpoint**: Cloud Foundry API endpoint.
* **Client ID/ Client Secret**: UAA client-id and client-secret.
The client needs authorities `cloud_controller.admin_read_only` and `doppler.firehose` and grant tpyes `client_credentials` and `refresh_token`.
Use the [uaac](https://github.com/cloudfoundry/cf-uaac) tool to create the appropriate client. The `uaac` commands to add a client are as follows:
<ol>
<li><code>uaac target https://uaa.[system domain url]</code></li>
<li> <code>uaac token client get admin -s [admin client credentials secret]</code></li>
<li> <code>uaac client add splunk-nozzle --name splunk-nozzle</code></li>
<li><code>uaac client add splunk-nozzle --secret [your_client_secret]</code></li>
<li> <code>uaac client add splunk-nozzle --authorized_grant_types client_credentials,refresh_token</code></li>
<li> <code>uaac client add splunk-nozzle --authorities doppler.firehose,cloud_controller.admin_read_only</code></li>
</ol>
* **Skip SSL validation**: Skip SSL certificate validation for connections to Cloud Foundry. Secure communications do not check SSL certificates against a trusted Certificate Authority.
Do not use for a production environment.
* **Event Types**: Choose which events the nozzle forwards to the Splunk indexer(s).
For more information about the following events, see
[dropsonde-protocol](https://github.com/cloudfoundry/dropsonde-protocol/tree/master/events)
* HttpStartStop
* LogMessage
* ValueMetric
* CounterEvent
* Error
* ContainerMetric
4. Click **Advanced**.
Complete the fields as follows:
* **Scale Out Nozzle**: Scale out Splunk Nozzle. Splunk recommends running two or more nozzles for high availability.
* **Firehose Subscription ID**: The Loggregator does a round-robin of events across connections
having the same ID. An operator would only need to change this if some other connection
is also using the default value.
* **Additional Fields**: Arbitrary set of key:value pairs in the form `key1:value1,key2:value2,key3:value3` that do not occur in event payload itself.
These fields are indexed with every event payload, and are used as metadata for indexing and searching.
One situation in which this might be useful is differentiating logs from multiple foundations going into the same Splunk Enterprise index.
* **Add App Information**: Enriches raw events with app metadata fields - app name, org name, org GUID, space name, space GUID.
When configured, the nozzle calls back to the Cloud Foundry API and queries events that
have an app GUID to add additional information. This causes a one-time load increase on the API machines proportional the number of running apps,
because each Nozzle populates a cache of app metadata.
* **Add Tags**: Add additional tags from envelope to splunk event. (Default: false) (Please note: Adding tags / Enabling this feature may slightly impact the performance due to the increased event size)
* **Enable Event Tracing**: Enables event trace logging.
Splunk events now contain a UUID, Splunk Nozzle Event Counts, and a Subscription-ID for Splunk correlation searches.
* **Monitor Queue Pressure**: Time interval (in s/m/h. For example, 3600s or 60m or 1h) for monitoring memory queue pressure. Use to help with back-pressure insights. (Increases CPU load. Use for insights purposes only) Default is 0s (Disabled).
* **HEC Retries**: The retry count for sending events to the Splunk platform.
Events not successfully sent after this number of retries will be dropped, causing data loss.
* **HEC Batch Size**: The number of events per batch sent to Splunk HTTP Event Collector.
* **HEC Workers**: The number of concurrent workers sending data to Splunk HTTP Event Collector.
Scale this number to your Splunk platform data collection capacity accordingly.
* **Consumer Queue Size**: The internal consumer queue buffer size. Events will be sent to your Splunk platform after queue is full.
* **Flush Interval**: Time interval (in s/m/h. For example, 3600s or 60m or 1h) for flushing queue to the Splunk platform regardless of Consumer Queue Size.
Prevents stale events in low throughput systems.
* **Missing App Cache Invalidate TTL**: Time interval (in s/m/h. For example, 3600s or 60m or 1h) between refreshing the missing app info cache.
Set to 0 to maintain cache until nozzle restart.
* **App Cache Invalidate TTL**: Time interval (in s/m/h. For example, 3600s or 60m or 1h) between refreshing the app info local cache.
Set to 0 to only populate the cache during startup or restart of the nozzle.
* **Org Space Cache Invalidate TTL**: Time interval (in s/m/h. For example, 3600s or 60m or 1h) between refreshing the org and space cache.
* **Ignore Missing App**: If the application is missing, then stop repeatedly querying application info from Cloud Foundry.
* **App Limits**: The number of apps for which metadata is gathered when refreshing the app metadata cache (order based on app creation date).
Set to 0 to remove limit.
* **Nozzle Memory**: Configure memory for Splunk Firehose Nozzle application in MB.
* **Firehose Keep Alive**: Keep alive duration (in s/m/h. For example, 3600s or 60m or 1h) for the firehose consumer.
* **Drop Warn Threshold**: Threshold for the count of dropped events in case the downstream is slow. Based on the threshold, the errors will be logged.
1. **Resource Config**: Select default options.
1. **Stemcell**: Ensure the proper stemcell is available.