Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable CI Visibility UI #221

Merged
merged 19 commits into from
Jun 29, 2021
Merged

Enable CI Visibility UI #221

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a9efbfa
Default configuration
drodriguezhdez Jun 14, 2021
a7fc2e2
Merge branch 'master' into drodriguezhdez/show_ciapp_UI
drodriguezhdez Jun 15, 2021
e3abfac
Edit UI for CI Visibility
drodriguezhdez Jun 15, 2021
0a5591b
Add help
drodriguezhdez Jun 22, 2021
888209c
Enable CI Visibility in the Jenkins plugin UI
drodriguezhdez Jun 22, 2021
4eeade3
Fix typo
drodriguezhdez Jun 22, 2021
f6bf735
Avoid throwing exception if CI Visibility config cannot be processed
drodriguezhdez Jun 22, 2021
5396f29
Accomodate the methods to the new CI Visibility properties
drodriguezhdez Jun 22, 2021
8f277fc
Apply suggestions from code review
drodriguezhdez Jun 23, 2021
37642a6
Apply suggestions from code review
drodriguezhdez Jun 23, 2021
3474585
Apply suggestions from code review
drodriguezhdez Jun 23, 2021
055f14c
Clearer conditions
drodriguezhdez Jun 23, 2021
a647e0a
Apply suggestions from code review
drodriguezhdez Jun 24, 2021
a73cd92
Apply feedback suggestions
drodriguezhdez Jun 24, 2021
47c98bb
Apply suggestions from code review
drodriguezhdez Jun 24, 2021
b9fd634
Apply suggestions from code review
drodriguezhdez Jun 25, 2021
70dbf6a
Include check connectivity CTA button
drodriguezhdez Jun 28, 2021
60ee64f
Change 'Connectivity' to 'Connection'
drodriguezhdez Jun 28, 2021
2d19d53
lowercase connectivity str
drodriguezhdez Jun 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 31 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,20 @@ This plugin can be installed from the [Update Center][3] (found at `Manage Jenki

There are two ways to configure your plugin to submit data to Datadog:

* **RECOMMENDED**: Using a DogStatsD server / Datadog Agent that acts as a forwarder between Jenkins and Datadog.
- Build Logs collection only works with a full Datadog Agent installed.
- For metrics submitted from an external host, the Datadog Agent requires the following configuration for DogStatsD: `dogstatsd_non_local_traffic: true`. This can be configured using the `datadog.yaml` [configuration file][17].
* **RECOMMENDED**: Using a Datadog Agent that acts as a forwarder between Jenkins and Datadog.
- When using a DogStatsD server instead of a full Datadog Agent, only metrics and events are supported.
- For data submitted from an external host, the Datadog Agent requires the following configuration: `dogstatsd_non_local_traffic: true` and `apm_non_local_traffic: true`. This can be configured using the `datadog.yaml` [configuration file][17].
* Sending data directly to Datadog through HTTP.
- The HTTP client implementation used is blocking with a timeout duration of 1 minute. If there is a connection problem with Datadog, it may slow your Jenkins instance down.
drodriguezhdez marked this conversation as resolved.
Show resolved Hide resolved
- This method does not currently support trace collection to be used with the "CI Visibility" product.

The configuration can be done from the [plugin user interface](#plugin-user-interface) with a [Groovy script](#groovy-script), or through [environment variables](#environment-variables).

#### Plugin user interface

To configure your Datadog Plugin, navigate to the `Manage Jenkins -> Configure System` page on your Jenkins installation. Once there, scroll down to find the `Datadog Plugin` section:

##### HTTP forwarding {#http-forwarding-plugin}
##### HTTP forwarding

1. Select the radio button next to **Use Datadog API URL and Key to report to Datadog** (selected by default).
2. Paste your [Datadog API key][4] in the `API Key` textbox on the Jenkins configuration screen.
Expand All @@ -49,19 +50,20 @@ To configure your Datadog Plugin, navigate to the `Manage Jenkins -> Configure S
5. (optional) Enter your [Datadog Log Intake URL][15] and select "Enable Log Collection" in the Advanced tab.
6. Save your configuration.

##### DogStatsD forwarding {#dogstatsd-forwarding-plugin}
##### Datadog Agent forwarding

1. Select the radio button next to **Use the Datadog Agent to report to Datadog**.
2. Specify your DogStatsD server `hostname` and `port`.
2. Specify your Datadog Agent `hostname` and `port`.
3. (optional) Enter the hostname of the Jenkins server in the Advanced tab to include it with the events.
4. (optional) Enter your Log Collection Port and configure [log collection](#log-collection-for-agents) and select "Enable Log Collection" in the Advanced tab.
4. (optional) Enter your Log Collection Port, configure [log collection](#log-collection-for-agents) in the Datadog Agent, and select "Enable Log Collection".
5. (optional) Enter your Trace Collection Port and select "Enable CI Visibility", optionally configuring your CI Instance name.
5. Save your configuration.

#### Groovy script

Configure your Datadog plugin to forward data through HTTP or DogStatsD using the Groovy scripts below. Configuring the plugin this way might be useful if you're running your Jenkins Master in a Docker container using the [official Jenkins Docker image][5] or any derivative that supports `plugins.txt` and Groovy init scripts.

##### HTTP forwarding {#http-forwarding-groovy-script}
##### HTTP forwarding using Groovy

```groovy
import jenkins.model.*
Expand All @@ -85,7 +87,7 @@ d.setLogIntakeUrl('https://http-intake.logs.datadoghq.com/v1/input/')
d.save()
```

##### DogStatsD forwarding {#dogstatsd-forwarding-groovy-script}
##### Datadog Agent forwarding using Groovy

```groovy
import jenkins.model.*
Expand All @@ -99,7 +101,13 @@ d.setTargetHost('localhost')
d.setTargetPort(8125)

// If you want to collect logs
d.setTargetLogCollectionPort(8125)
d.setTargetLogCollectionPort(10518)
d.setCollectBuildLogs(true)

// If you want to enable CI Visibility
d.setTargetTraceCollectionPort(8126)
d.setEnableCiVisibility(true)
d.setCiInstanceName("jenkins")

// Customization, see dedicated section below
d.setExcluded('job1,job2')
Expand All @@ -112,19 +120,26 @@ d.save()

Configure your Datadog plugin using environment variables with the `DATADOG_JENKINS_PLUGIN_REPORT_WITH` variable, which specifies the report mechanism to use.

##### HTTP forwarding {#http-forwarding-env}
##### HTTP forwarding using environment variables

1. Set the `DATADOG_JENKINS_PLUGIN_REPORT_WITH` variable to `HTTP`.
2. Set the `DATADOG_JENKINS_PLUGIN_TARGET_API_URL` variable, which specifies the Datadog API endpoint (defaults to `https://api.datadoghq.com/api/`).
3. Set the `DATADOG_JENKINS_PLUGIN_TARGET_API_KEY` variable, which specifies your [Datadog API key][4].
4. (optional) Set the `DATADOG_JENKINS_PLUGIN_TARGET_LOG_INTAKE_URL` variable, which specifies the Datadog Log Intake URL (defaults to `https://http-intake.logs.datadoghq.com/v1/input/`).

##### DogStatsD forwarding {#dogstatsd-forwarding-env}
##### Datadog Agent forwarding using environment variables

1. Set the `DATADOG_JENKINS_PLUGIN_REPORT_WITH` variable to `DSD`.
2. Set the `DATADOG_JENKINS_PLUGIN_TARGET_HOST` variable, which specifies the DogStatsD server host (defaults to `localhost`).
3. Set the `DATADOG_JENKINS_PLUGIN_TARGET_PORT` variable, which specifies the DogStatsD server port (defaults to `8125`).
4. (optional) Set the `DATADOG_JENKINS_PLUGIN_TARGET_LOG_COLLECTION_PORT` variable, which specifies the Datadog Agent log collection port.
4. (optional) Log Collection:
- Enable [log collection](#log-collection-for-agents) in the Datadog Agent.
- Set the `DATADOG_JENKINS_PLUGIN_TARGET_LOG_COLLECTION_PORT` variable, which specifies the Datadog Agent log collection port.
drodriguezhdez marked this conversation as resolved.
Show resolved Hide resolved
- Set the `DATADOG_JENKINS_PLUGIN_COLLECT_BUILD_LOGS` variable to `true` in order to enable log collection (disabled by default).
5. (optional) CI Visibility (trace collection):
- Set the `DATADOG_JENKINS_PLUGIN_TARGET_TRACE_COLLECTION_PORT` variable, which specifies the Datadog Agent trace collection port (defaults to `8126`).
- Set the `DATADOG_JENKINS_PLUGIN_ENABLE_CI_VISIBILITY` variable to `true` in order to enable CI Visibility (disabled by default).
- Set the `DATADOG_JENKINS_PLUGIN_CI_VISIBILITY_CI_INSTANCE_NAME` variable, which specifies the name of the Jenkins instance for CI Visibility (defaults to `jenkins`).

#### Logging

Expand Down Expand Up @@ -171,14 +186,14 @@ To customize your global configuration, in Jenkins navigate to `Manage Jenkins -

| Customization | Description | Environment variable |
|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------|
| Excluded jobs | A comma-separated list of regex used to exclude job names from monitoring, for example: `susans-job,johns-.*,prod_folder/prod_release`. | `DATADOG_JENKINS_PLUGIN_EXCLUDED` |
| Included jobs | A comma-separated list of regex used to include job names for monitoring, for example: `susans-job,johns-.*,prod_folder/prod_release`. | `DATADOG_JENKINS_PLUGIN_INCLUDED` |
| Hostname | A hostname to use in every event sent to Datadog. | `DATADOG_JENKINS_PLUGIN_HOSTNAME` |
drodriguezhdez marked this conversation as resolved.
Show resolved Hide resolved
| Excluded jobs | A comma-separated list of regex used to exclude job names from monitoring, for example: `susans-job,johns-.*,prod_folder/prod_release`. | `DATADOG_JENKINS_PLUGIN_EXCLUDED` |
| Included jobs | A comma-separated list of regex used to include job names for monitoring, for example: `susans-job,johns-.*,prod_folder/prod_release`. | `DATADOG_JENKINS_PLUGIN_INCLUDED` |
| Global tag file | The path to a workspace file containing a comma separated list of tags (not compatible with pipeline jobs). | `DATADOG_JENKINS_PLUGIN_GLOBAL_TAG_FILE` |
| Global tags | A comma-separated list of tags to apply to all metrics, events, and service checks. Tags can include environment variables that are defined in the master jenkins instance. | `DATADOG_JENKINS_PLUGIN_GLOBAL_TAGS` |
| Global job tags | A comma separated list of regex to match a job and a list of tags to apply to that job. Tags can include environment variables that are defined in the master jenkins instance. **Note**: Tags can reference match groups in the regex using the `$` symbol, for example: `(.*?)_job_(*?)_release, owner:$1, release_env:$2, optional:Tag3` | `DATADOG_JENKINS_PLUGIN_GLOBAL_JOB_TAGS` |
| Send security audit events | Submits the `Security Events Type` of events and metrics (enabled by default). | `DATADOG_JENKINS_PLUGIN_EMIT_SECURITY_EVENTS` |
| Send system events | Submits the `System Events Type` of events and metrics (enabled by default). | `DATADOG_JENKINS_PLUGIN_EMIT_SYSTEM_EVENTS` |
| Enable Log Collection | Collect and Submit build logs (disabled by default). | `DATADOG_JENKINS_PLUGIN_COLLECT_BUILD_LOGS` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you removing this ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not removing this, but moving to another place in the README. These envvars are related to the Advanced tab, where the "Enable Log Collection" is not there anymore.


### Job customization

Expand Down
Loading