Skip to content

Commit

Permalink
Remove bucket_patterns param
Browse files Browse the repository at this point in the history
This can be achieved in prometheus with a relabel_config.
  • Loading branch information
ribbybibby committed Jun 26, 2021
1 parent d42b01a commit f0f1d7e
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 25 deletions.
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,35 +116,37 @@ exporter has access to.

This should be all the config required to successfully scrape every bucket:

```
```yml
scrape_configs:
- job_name: 's3'
- job_name: "s3"
metrics_path: /probe
http_sd_configs:
- url: http://127.0.0.1:9340/discovery
```

You can limit the buckets returned with the `bucket_pattern` parameter. Refer to
the documentation for [`path.Match`](https://golang.org/pkg/path/#Match) for the
pattern syntax.
Use `relabel_configs` to select the buckets you want to scrape:

```
```yml
scrape_configs:
- job_name: 's3'
- job_name: "s3"
metrics_path: /probe
http_sd_configs:
# This will only discover buckets with a name that starts with example-
- url: http://127.0.0.1:9340/discovery?bucket_pattern=example-*
- url: http://127.0.0.1:9340/discovery
relabel_configs:
# Keep buckets that start with example-
- source_labels: [__param_bucket]
action: keep
regex: ^example-.*
```

The prefix can be set too, but be mindful that this will apply to all buckets:

```
```yml
scrape_configs:
- job_name: 's3'
- job_name: "s3"
metrics_path: /probe
http_sd_configs:
- url: http://127.0.0.1:9340/discovery?bucket_pattern=example-*
- url: http://127.0.0.1:9340/discovery
params:
prefix: ["thing.txt"]
```
Expand Down
15 changes: 2 additions & 13 deletions s3_exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"encoding/json"
"net/http"
"os"
"path"
"time"

"github.com/prometheus/client_golang/prometheus"
Expand Down Expand Up @@ -173,27 +172,17 @@ type discoveryTarget struct {
}

func discoveryHandler(w http.ResponseWriter, r *http.Request, svc s3iface.S3API) {
// If a bucket pattern isn't specified, then return every bucket
bucketPattern := r.URL.Query().Get("bucket_pattern")
if bucketPattern == "" {
bucketPattern = "*"
}

result, err := svc.ListBuckets(&s3.ListBucketsInput{})
if err != nil {
log.Errorln(err)
http.Error(w, "error listing buckets", http.StatusInternalServerError)
return
}

targets := []discoveryTarget{}
for _, b := range result.Buckets {
name := aws.StringValue(b.Name)

matched, err := path.Match(bucketPattern, name)
if err != nil {
http.Error(w, "bad pattern provided for 'bucket_pattern'", http.StatusBadRequest)
}
if matched {
if name != "" {
t := discoveryTarget{
Targets: []string{r.Host},
Labels: map[string]string{
Expand Down

0 comments on commit f0f1d7e

Please sign in to comment.