Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix point-in-time Prometheus metrics. #4948

Merged
merged 7 commits into from
Mar 19, 2020

Conversation

danielmai
Copy link
Contributor

@danielmai danielmai commented Mar 17, 2020

Fixes #4532.

Changed metrics:

  • dgraph_pending_queries_total
  • dgraph_active_mutations_total
  • dgraph_alpha_health_status
  • dgraph_pending_proposals_total

The metrics for pending queries and active mutations would report "1" or "-1" because the metrics view was set to LastValue. This change fixes this by changing the view to a Sum so measurements of "1" and "-1" would accumulate the metrics as expected.

There were also metrics whose values were separated by tags when they should be treated as a single metric. This made metrics that were incremented and decremented show up as seperate metrics. For these metrics, the tags are excluded from the view so the metrics are shown correctly:

Example

Before:

dgraph_pending_queries_total{method="Server.Query",status=""} 100
dgraph_pending_queries_total{method="Server.Query",status="ok"} -100

After:

dgraph_pending_queries_total 0

This change is Reviewable

@danielmai danielmai requested review from manishrjain and a team as code owners March 17, 2020 16:36
Copy link
Contributor

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the build issue but otherwise it :lgtm:

Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @danielmai and @manishrjain)


edgraph/server.go, line 805 at r1 (raw file):

	}

	isGraphQL, _ := ctx.Value(isGraphQL).(bool)

Build is failing with: ../edgraph/server.go:805:15: no new variables on left side of :=

PendingProposals now measures the current IOU count.
Copy link
Contributor Author

@danielmai danielmai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @manishrjain and @martinmr)


edgraph/server.go, line 805 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

Build is failing with: ../edgraph/server.go:805:15: no new variables on left side of :=

Done.

@danielmai danielmai changed the title Fix point-in-time metrics view. Fix point-in-time Prometheus metrics view. Mar 18, 2020
@danielmai danielmai changed the title Fix point-in-time Prometheus metrics view. Fix point-in-time Prometheus metrics. Mar 18, 2020
@danielmai danielmai merged commit 6538fce into master Mar 19, 2020
@danielmai danielmai deleted the danielmai/fix-point-in-time-metrics branch March 19, 2020 04:08
danielmai added a commit that referenced this pull request Mar 19, 2020
Fixes #4532.

The metrics for pending queries and active mutations would
report "1" or "-1" because the metrics view was set to LastValue.
This change fixes this by changing the view to a Sum so
measurements of "1" and "-1" would accumulate the metrics as
expected.

There were also metrics whose values were separated by tags when
they should be treated as a single metric. This made metrics that
were incremented and decremented show up as seperate metrics,
e.g.:

dgraph_pending_queries_total{method="Server.Query",status=""} 100
dgraph_pending_queries_total{method="Server.Query",status="ok"} -100

For these metrics, the tags are excluded from the view so the
metrics are shown correctly:

dgraph_pending_queries_total 0
danielmai added a commit that referenced this pull request Mar 19, 2020
(cherry-pick of #4948)

Fixes #4532.

The metrics for pending queries and active mutations would
report "1" or "-1" because the metrics view was set to LastValue.
This change fixes this by changing the view to a Sum so
measurements of "1" and "-1" would accumulate the metrics as
expected.

There were also metrics whose values were separated by tags when
they should be treated as a single metric. This made metrics that
were incremented and decremented show up as seperate metrics,
e.g.:

dgraph_pending_queries_total{method="Server.Query",status=""} 100
dgraph_pending_queries_total{method="Server.Query",status="ok"} -100

For these metrics, the tags are excluded from the view so the
metrics are shown correctly:

dgraph_pending_queries_total 0
danielmai added a commit that referenced this pull request Mar 20, 2020
Fixes #4532.

The metrics for pending queries and active mutations would
report "1" or "-1" because the metrics view was set to LastValue.
This change fixes this by changing the view to a Sum so
measurements of "1" and "-1" would accumulate the metrics as
expected.

There were also metrics whose values were separated by tags when
they should be treated as a single metric. This made metrics that
were incremented and decremented show up as seperate metrics,
e.g.:

dgraph_pending_queries_total{method="Server.Query",status=""} 100
dgraph_pending_queries_total{method="Server.Query",status="ok"} -100

For these metrics, the tags are excluded from the view so the
metrics are shown correctly:

dgraph_pending_queries_total 0

(cherry picked from commit 6538fce)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Dgraph is not recording metrics correctly
2 participants