metrics service sink: add config option to report counters as deltas #10889
Conversation
Signed-off-by: Jose Nino <jnino@lyft.com>
Signed-off-by: Jose Nino <jnino@lyft.com>
Signed-off-by: Jose Nino <jnino@lyft.com>
|
Additional context: Envoy Mobile has been using the metrics service sink to gather data since mid-november. However, in the low volume environments we were testing we did not notice that counters were being amplified. In the last couple of weeks we started testing in production, and I noticed that our values for counters were orders of magnitude higher than expected (and cross-validated with client events). I noticed the problem is that the metrics service sink reports the current value of counters on a flush, rather than the delta. Given the timestamp is reported with the metric we could solve for this on the receiving service. However, given that Envoy already has the needed information, I found it more efficient to have the sink report the delta. |
source/extensions/stat_sinks/metrics_service/grpc_metrics_service_impl.h
Show resolved
Hide resolved
Signed-off-by: Jose Nino <jnino@lyft.com>
|
@htuch @ramaraochavali I have created an issue to track future work #10968 and merged master. Do you mind doing another pass? |
Signed-off-by: Jose Nino <jnino@lyft.com>
|
unassigning myself as I see @htuch is on it. |
test/extensions/stats_sinks/metrics_service/grpc_metrics_service_impl_test.cc
Show resolved
Hide resolved
Description: updates the envoy ref. Brings in the ability to report counters as deltas in the metrics service stats sink envoyproxy/envoy#10889. Signed-off-by: Jose Nino <jnino@lyft.com>
Description: updates the envoy ref. Brings in the ability to report counters as deltas in the metrics service stats sink #10889. Signed-off-by: Jose Nino <jnino@lyft.com> Signed-off-by: JP Simard <jp@jpsim.com>
Description: updates the envoy ref. Brings in the ability to report counters as deltas in the metrics service stats sink #10889. Signed-off-by: Jose Nino <jnino@lyft.com> Signed-off-by: JP Simard <jp@jpsim.com>
Description: this PR adds the ability to configure the metrics service stats sink to report counters as deltas between flushing intervals. This is the expected representation for some stats aggregations backends. Similar behavior is seen, for instance, in the statsd sink.
Risk Level: low, previous behavior is left unchanged, and is the default.
Testing: updated unit test.
Docs Changes: left comments in the field definition
Release Notes: updated version history.
Signed-off-by: Jose Nino jnino@lyft.com