-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add vts-sum-key config flag #2356
Conversation
34fadef
to
8b8a249
Compare
Codecov Report
@@ Coverage Diff @@
## master #2356 +/- ##
==========================================
+ Coverage 39.11% 39.12% +0.01%
==========================================
Files 73 73
Lines 5203 5204 +1
==========================================
+ Hits 2035 2036 +1
Misses 2878 2878
Partials 290 290
Continue to review full report at Codecov.
|
Okay, I'll give it a try: /assign @dcbw |
8b8a249
to
8886b8a
Compare
Updated to resolve bindata conflict. |
/lgtm |
/lgtm |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aledbf, gianrubio, nicknovitski The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@gianrubio thanks for the review. |
Currently, vts metrics use the server zone key
*
to indicate "all other server zones combined." The prometheus exporter translates this into a label with the same value. I use a datadog agent to read the prometheus metrics, which turns the label into a "tag", but strips out characters which the datadog service cannot accept, including*
. This means the label value*
becomes an empty string tag value.So for example the prometheus metric
nginx_requests_total{ingress_class="nginx",namespace="",server_zone="*"}
is turned into a datadog metric namednginx.requests_total
, with the tagsingress_class:nginx
,namespace:
, andserver_zone:
.This isn't so bad, except the same thing happens to metrics for the server zone which includes (I believe) the default backend and metrics server, labeled
server_zone="_"
. So I have different metrics numbers with the same tags, getting sent to datadog all the time, which makes monitoring seemingly impossible.The workaround I've done is to create a custom
nginx.conf
template exactly like the normal one, except with one additional line:vhost_traffic_status_display_sum_key total;
And maybe someday I'll just use prometheus instead of datadog. But in the meantime, I thought this would be a nice feature to have.