Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GEN-1947]: add Better Stack destination support and documentation #2083

Merged
merged 11 commits into from
Dec 29, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 43 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ Odigos is an open-source distributed tracing solution that simplifyes and improv

## Key Features

* **Code-Free Instrumentation** : Set up distributed tracing in minutes, eliminating manual code modifications.
* **Multi-Language Support** : Works with Java, Python, .NET, Node.js, and Go applications.
* **eBPF-Powered** : Utilizes eBPF technology for high-performance instrumentation of Go applications. eBPF-based instrumentation for Java, Python, and Node.js is available in the enterprise edition.
* **OpenTelemetry Compatible** : Generates traces in OpenTelemetry format for broad tool compatibility.
* **Vendor Agnostic** : Integrates with various monitoring solutions, avoiding vendor lock-in.
* **Automatic Scaling** : Manages and scales OpenTelemetry collectors based on data volume.
* **Opinionated Defaults** : Supplies common defaults and best practices out-of-the-box, requiring no deep knowledge of OpenTelemetry.
- **Code-Free Instrumentation** : Set up distributed tracing in minutes, eliminating manual code modifications.
- **Multi-Language Support** : Works with Java, Python, .NET, Node.js, and Go applications.
- **eBPF-Powered** : Utilizes eBPF technology for high-performance instrumentation of Go applications. eBPF-based instrumentation for Java, Python, and Node.js is available in the enterprise edition.
- **OpenTelemetry Compatible** : Generates traces in OpenTelemetry format for broad tool compatibility.
- **Vendor Agnostic** : Integrates with various monitoring solutions, avoiding vendor lock-in.
- **Automatic Scaling** : Manages and scales OpenTelemetry collectors based on data volume.
- **Opinionated Defaults** : Supplies common defaults and best practices out-of-the-box, requiring no deep knowledge of OpenTelemetry.

## Why Choose Odigos

Expand Down Expand Up @@ -85,48 +85,47 @@ For more details, see our [quickstart guide](https://docs.odigos.io/intro).

### Managed Destinations


| Destination | Traces | Metrics | Logs |
| ------------------------- | :------: | :-------: | :----: |
| AppDynamics | ✅ | ✅ | ✅ |
| Axiom | ✅ | | ✅ |
| AWS S3 | ✅ | | ✅ |
| Azure Blob Storage | ✅ | | ✅ |
| Causely | ✅ | | |
| Chronosphere | ✅ | ✅ | |
| Coralogix | ✅ | ✅ | ✅ |
| Datadog | ✅ | ✅ | ✅ |
| Dynatrace | ✅ | ✅ | ✅ |
| Gigapipe | ✅ | | |
| Google Cloud Monitoring | ✅ | ✅ | |
| Google Cloud Storage | ✅ | | ✅ |
| Grafana Cloud | ✅ | ✅ | ✅ |
| Honeycomb | ✅ | ✅ | ✅ |
| Last9 | ✅ | ✅ | |
| Lightstep | ✅ | | |
| Logz.io | ✅ | ✅ | ✅ |
| New Relic | ✅ | ✅ | ✅ |
| OpsVerse | ✅ | ✅ | ✅ |
| Sentry | ✅ | | |
| Splunk | ✅ | | |
| Sumo Logic | ✅ | ✅ | ✅ |
| ----------------------- | :----: | :-----: | :--: |
| AppDynamics | ✅ | ✅ | ✅ |
| Axiom | ✅ | | ✅ |
| AWS S3 | ✅ | | ✅ |
| Azure Blob Storage | ✅ | | ✅ |
| Better Stack | | ✅ | ✅ |
| Causely | ✅ | | |
| Chronosphere | ✅ | ✅ | |
| Coralogix | ✅ | ✅ | ✅ |
| Datadog | ✅ | ✅ | ✅ |
| Dynatrace | ✅ | ✅ | ✅ |
| Gigapipe | ✅ | | |
| Google Cloud Monitoring | ✅ | ✅ | |
| Google Cloud Storage | ✅ | | ✅ |
| Grafana Cloud | ✅ | ✅ | ✅ |
| Honeycomb | ✅ | ✅ | ✅ |
| Last9 | ✅ | ✅ | |
| Lightstep | ✅ | | |
| Logz.io | ✅ | ✅ | ✅ |
| New Relic | ✅ | ✅ | ✅ |
| OpsVerse | ✅ | ✅ | ✅ |
| Sentry | ✅ | | |
| Splunk | ✅ | | |
| Sumo Logic | ✅ | ✅ | ✅ |

## Self-Hosted (Open Source) Destinations


| Destination | Traces | Metrics | Logs |
| --------------- | :------: | :-------: | :----: |
| ClickHouse | ✅ | ✅ | ✅ |
| Elasticsearch | ✅ | | ✅ |
| Jaeger | ✅ | | |
| Loki | | | ✅ |
| OTLP | ✅ | ✅ | ✅ |
| OTLP HTTP | ✅ | ✅ | ✅ |
| Prometheus | | ✅ | |
| Quickwit | ✅ | | |
| qryn | ✅ | ✅ | ✅ |
| SigNoz | ✅ | ✅ | ✅ |
| Tempo | ✅ | | |
| ------------- | :----: | :-----: | :--: |
| ClickHouse | ✅ | ✅ | ✅ |
| Elasticsearch | ✅ | | ✅ |
| Jaeger | ✅ | | |
| Loki | | | ✅ |
| OTLP | ✅ | ✅ | ✅ |
| OTLP HTTP | ✅ | ✅ | ✅ |
| Prometheus | | ✅ | |
| Quickwit | ✅ | | |
| qryn | ✅ | ✅ | ✅ |
| SigNoz | ✅ | ✅ | ✅ |
| Tempo | ✅ | | |

Can't find the destination you need? Help us by following our quick [add new destination](https://docs.odigos.io/adding-new-dest) guide and submitting a PR.

Expand Down
87 changes: 87 additions & 0 deletions common/config/betterstack.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
package config

import (
"errors"

"github.com/odigos-io/odigos/common"
)

const (
BetterStackMetricsEndpoint = "BETTERSTACK_METRICS_ENDPOINT"
BetterStackLogsEndpoint = "BETTERSTACK_LOGS_ENDPOINT"
)

var (
ErrorBetterStackMetricsEndpointMissing = errors.New("BetterStack is missing a required field (\"BETTERSTACK_METRICS_ENDPOINT\"), BetterStack will not be configured")
ErrorBetterStackLogsEndpointMissing = errors.New("BetterStack is missing a required field (\"BETTERSTACK_LOGS_ENDPOINT\"), BetterStack will not be configured")
)

type BetterStack struct{}

func (j *BetterStack) DestType() common.DestinationType {
return common.BetterStackDestinationType
}

func (j *BetterStack) ModifyConfig(dest ExporterConfigurer, cfg *Config) error {
config := dest.GetConfig()
uniqueUri := "betterstack-" + dest.GetID()

processorName := "attributes/betterstack"
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
cfg.Processors[processorName] = GenericMap{
"actions": []GenericMap{
{
"key": "better_stack_source_token",
"value": "${BETTERSTACK_TOKEN}",
"action": "insert",
},
},
}

if isMetricsEnabled(dest) {
url, exists := config[BetterStackMetricsEndpoint]
if !exists {
return ErrorBetterStackMetricsEndpointMissing
}

endpoint, err := parseOtlpHttpEndpoint(url)
if err != nil {
return err
}

exporterName := "prometheusremotewrite/" + uniqueUri
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
cfg.Exporters[exporterName] = GenericMap{
"endpoint": endpoint,
}

pipeName := "metrics/" + uniqueUri
cfg.Service.Pipelines[pipeName] = Pipeline{
Processors: []string{processorName},
Exporters: []string{exporterName},
}
}

if isLoggingEnabled(dest) {
url, exists := config[BetterStackLogsEndpoint]
if !exists {
return ErrorBetterStackLogsEndpointMissing
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
}

endpoint, err := parseOtlpHttpEndpoint(url)
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return err
}

exporterName := "otlp/" + uniqueUri
cfg.Exporters[exporterName] = GenericMap{
"endpoint": endpoint,
}

pipeName := "logs/" + uniqueUri
cfg.Service.Pipelines[pipeName] = Pipeline{
Processors: []string{processorName},
Exporters: []string{exporterName},
}
}

return nil
}
12 changes: 6 additions & 6 deletions common/config/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ const (
)

var availableConfigers = []Configer{
&Middleware{}, &AppDynamics{}, &Honeycomb{}, &GrafanaCloudPrometheus{}, &GrafanaCloudTempo{},
&GrafanaCloudLoki{}, &Datadog{}, &NewRelic{}, &Logzio{}, &Last9{}, &Prometheus{},
&Tempo{}, &Loki{}, &Jaeger{}, &GenericOTLP{}, &OTLPHttp{}, &Elasticsearch{}, &Quickwit{}, &Signoz{}, &Qryn{},
&OpsVerse{}, &Splunk{}, &Lightstep{}, &GoogleCloud{}, &GoogleCloudStorage{}, &Sentry{}, &AzureBlobStorage{},
&AWSS3{}, &Dynatrace{}, &Chronosphere{}, &ElasticAPM{}, &Axiom{}, &SumoLogic{}, &Coralogix{}, &Clickhouse{},
&Causely{}, &Uptrace{}, &Debug{}, &QrynOSS{}, &Nop{}, &Mock{},
&AppDynamics{}, &Axiom{}, &AWSS3{}, &AzureBlobStorage{}, &BetterStack{}, &Causely{}, &Chronosphere{}, &Clickhouse{}, &Coralogix{},
&Datadog{}, &Debug{}, &Dynatrace{}, &ElasticAPM{}, &Elasticsearch{}, &GenericOTLP{}, &GoogleCloud{},
&GoogleCloudStorage{}, &GrafanaCloudLoki{}, &GrafanaCloudPrometheus{}, &GrafanaCloudTempo{},
&Honeycomb{}, &Jaeger{}, &Last9{}, &Lightstep{}, &Logzio{}, &Loki{}, &Middleware{}, &Mock{}, &NewRelic{},
&Nop{}, &OpsVerse{}, &OTLPHttp{}, &Prometheus{}, &Qryn{}, &QrynOSS{}, &Quickwit{}, &Sentry{},
&Signoz{}, &Splunk{}, &SumoLogic{}, &Tempo{}, &Uptrace{},
}

type Configer interface {
Expand Down
1 change: 1 addition & 0 deletions common/dests.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ const (
AWSS3DestinationType DestinationType = "s3"
AxiomDestinationType DestinationType = "axiom"
AzureBlobDestinationType DestinationType = "azureblob"
BetterStackDestinationType DestinationType = "betterstack"
CauselyDestinationType DestinationType = "causely"
ChronosphereDestinationType DestinationType = "chronosphere"
ClickhouseDestinationType DestinationType = "clickhouse"
Expand Down
37 changes: 37 additions & 0 deletions destinations/data/betterstack.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
apiVersion: internal.odigos.io/v1beta1
kind: Destination
metadata:
type: betterstack
displayName: Better Stack
category: managed
spec:
image: betterstack.svg
signals:
traces:
supported: false
metrics:
supported: true
logs:
supported: true
fields:
- name: BETTERSTACK_METRICS_ENDPOINT
displayName: OTLP HTTP Metrics Endpoint
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
componentType: input
initialValue: 'https://in-otel.logs.betterstack.com/metrics'
componentProps:
type: text
required: true
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
- name: BETTERSTACK_LOGS_ENDPOINT
displayName: OTLP HTTP Logs Endpoint
BenElferink marked this conversation as resolved.
Show resolved Hide resolved
componentType: input
initialValue: 'https://in-otel.logs.betterstack.com:443'
componentProps:
type: text
required: true
- name: BETTERSTACK_TOKEN
displayName: Better Stack Source Token
componentType: input
secret: true
componentProps:
type: password
required: true
8 changes: 8 additions & 0 deletions destinations/logos/betterstack.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/backends-overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Odigos has destinations for many observability backends.
| Axiom | Managed | ✅ | | ✅ |
| AWS S3 | Managed | ✅ | | ✅ |
| Azure Blob Storage | Managed | ✅ | | ✅ |
| Better Stack | Managed | | ✅ | ✅ |
| Causely | Managed | ✅ | | |
| Chronosphere | Managed | ✅ | ✅ | |
| ClickHouse | Self-Hosted | ✅ | ✅ | ✅ |
Expand Down
59 changes: 59 additions & 0 deletions docs/backends/betterstack.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: 'Better Stack'
---

## Configuring Backend

- **BETTERSTACK_METRICS_ENDPOINT** - OpenTelemetry HTTP Metrics Endpoint, defaults to `https://in-otel.logs.betterstack.com/metrics`.
- **BETTERSTACK_LOGS_ENDPOINT** - OpenTelemetry HTTP Logs Endpoint, defaults to `https://in-otel.logs.betterstack.com:443`.
- **BETTERSTACK_TOKEN** - Source Token, generated from Better Stack UI.

## Adding a Destination to Odigos

Odigos makes it simple to add and configure destinations, allowing you to select the specific signals [traces/logs/metrics] that you want to send to each destination. There are two primary methods for configuring destinations in Odigos:

1. **Using the UI**

Use the [Odigos CLI](https://docs.odigos.io/cli/odigos_ui) to access the UI:

```bash
odigos ui
```

2. **Using kubernetes manifests**

Save the YAML below to a file (e.g., `destination.yaml`) and apply it using `kubectl`:

```bash
kubectl apply -f destination.yaml
```


```yaml
apiVersion: odigos.io/v1alpha1
kind: Destination
metadata:
name: betterstack-example
namespace: odigos-system
spec:
data:
BETTERSTACK_LOGS_ENDPOINT: <OTLP HTTP Logs Endpoint>
BETTERSTACK_METRICS_ENDPOINT: <OTLP HTTP Metrics Endpoint>
destinationName: betterstack
secretRef:
name: betterstack-secret
signals:
- METRICS
- LOGS
type: betterstack

---
apiVersion: v1
data:
BETTERSTACK_TOKEN: <base64 Better Stack Source Token>
kind: Secret
metadata:
name: betterstack-secret
namespace: odigos-system
type: Opaque
```
Loading
Loading