Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record aggregator metrics in event store #2023

Closed
9 tasks done
Tracked by #2028
jpraynaud opened this issue Oct 21, 2024 · 0 comments · Fixed by #2059
Closed
9 tasks done
Tracked by #2028

Record aggregator metrics in event store #2023

jpraynaud opened this issue Oct 21, 2024 · 0 comments · Fixed by #2059
Assignees
Labels
monitoring 👀 Monitoring / Alerting

Comments

@jpraynaud
Copy link
Member

jpraynaud commented Oct 21, 2024

Why

We need to store usage metrics in the event store of the aggregator in order to produce statistics reports. We want this feature to have a low impact on the performances of the aggregator.

What

Use the already existing Metrics service to create associated events (aggregated counter values) to be recorded in the existing event store at regular pace.

How

  • Use the same migration system in the event store database than in other aggregator stores (in a different PR)
  • Make the Metrics service export its counters in a HashMap<String, u32>
  • Create a new UsageReporter service in the aggregator
    • Records delta of Metrics service counters between now and last call at regular pace
    • Produce events to be stored the event store when the delta of a counter is strictly greater than 0
    • Each event needs to embed the period used to compute the delta of the counter
    • Make the pace of reporting a configuration option of the aggregator
  • Create a new runbook and SQL view for retrieval of:
    • Total counter values split by day (each row has date, counter_name, value)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
monitoring 👀 Monitoring / Alerting
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants