Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add multiplier option to prometheus helper #10994

Merged
merged 2 commits into from
Mar 6, 2019
Merged

Add multiplier option to prometheus helper #10994

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f1dd013
Add multiplier option to prometheus helper
jsoriano Feb 28, 2019
7636df2
Add changelog entry
jsoriano Feb 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Update haproxy.* fields to map to ECS. {pull}10558[10558] {pull}10568[10568]
- Collect all EC2 meta data from all instances in all states. {pull}10628[10628]
- Migrate docker module to ECS. {pull}10927[10927]
- Add new option `OpMultiplyBuckets` to scale histogram buckets to avoid decimal points in final events {pull}10994[10994]

*Packetbeat*

Expand Down
34 changes: 34 additions & 0 deletions metricbeat/helper/prometheus/metric.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,13 @@ func OpLowercaseValue() MetricOption {
return opLowercaseValue{}
}

// OpMultiplyBuckets multiplies bucket labels in histograms, useful to change units
func OpMultiplyBuckets(multiplier float64) MetricOption {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd call this NewOpMultiplyBucketsOption unless we have some internal convention for this

Copy link
Member Author

@jsoriano jsoriano Mar 6, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have an strong opinion on this but yes, I am following the naming schemas used here.

Take into account that they are not so bad as they are used as options in prometheus metrics:

Metrics: map[string]MetricMap{
    "histogram_decimal_metric": Metric("histogram.metric", OpMultiplyBuckets(1000))
}

return opMultiplyBuckets{
multiplier: multiplier,
}
}

// Metric directly maps a Prometheus metric to a Metricbeat field
func Metric(field string, options ...MetricOption) MetricMap {
return &commonMetric{
Expand Down Expand Up @@ -267,3 +274,30 @@ func (o opLowercaseValue) Process(field string, value interface{}, labels common
}
return field, value, labels
}

type opMultiplyBuckets struct {
multiplier float64
}

// Process will multiply the bucket labels if it is an histogram with numeric labels
func (o opMultiplyBuckets) Process(field string, value interface{}, labels common.MapStr) (string, interface{}, common.MapStr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for not using pointers here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I didn't think about this, I just followed what was being done for other options.

In any case I wouldn't change it now, we can consider doing it for all options in the future.

histogram, ok := value.(common.MapStr)
if !ok {
return field, value, labels
}
bucket, ok := histogram["bucket"].(common.MapStr)
if !ok {
return field, value, labels
}
multiplied := common.MapStr{}
for k, v := range bucket {
if f, err := strconv.ParseFloat(k, 64); err == nil {
key := strconv.FormatFloat(f*o.multiplier, 'f', -1, 64)
multiplied[key] = v
} else {
multiplied[k] = v
}
}
histogram["bucket"] = multiplied
return field, histogram, labels
}
33 changes: 33 additions & 0 deletions metricbeat/helper/prometheus/prometheus_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ histogram_metric_bucket{le="1e+09"} 1
histogram_metric_bucket{le="+Inf"} 1
histogram_metric_sum 117
histogram_metric_count 1
# TYPE histogram_decimal_metric histogram
histogram_decimal_metric_bucket{le="0.001"} 1
histogram_decimal_metric_bucket{le="0.01"} 1
histogram_decimal_metric_bucket{le="0.1"} 2
histogram_decimal_metric_bucket{le="1"} 3
histogram_decimal_metric_bucket{le="+Inf"} 5
histogram_decimal_metric_sum 4.31
histogram_decimal_metric_count 5

`

Expand Down Expand Up @@ -329,6 +337,31 @@ func TestPrometheus(t *testing.T) {
},
},
},
{
msg: "Histogram decimal metric",
mapping: &MetricsMapping{
Metrics: map[string]MetricMap{
"histogram_decimal_metric": Metric("histogram.metric", OpMultiplyBuckets(1000)),
},
},
expected: []common.MapStr{
common.MapStr{
"histogram": common.MapStr{
"metric": common.MapStr{
"count": uint64(5),
"bucket": common.MapStr{
"1": uint64(1),
"10": uint64(1),
"100": uint64(2),
"1000": uint64(3),
"+Inf": uint64(5),
},
"sum": 4.31,
},
},
},
},
},
}

for _, test := range tests {
Expand Down