Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define the sizing rule for Loki volume #2687

Closed
alexandre-allard opened this issue Jul 29, 2020 · 0 comments · Fixed by #2799
Closed

Define the sizing rule for Loki volume #2687

alexandre-allard opened this issue Jul 29, 2020 · 0 comments · Fixed by #2799
Assignees
Labels
complexity:medium Something that requires one or few days to fix topic:docs Documentation topic:log Anything related to log centralization system

Comments

@alexandre-allard
Copy link
Contributor

alexandre-allard commented Jul 29, 2020

Component: loki, documentation

Why this is needed:
To provide a simple formula to help calculate the ideal size needed for the Loki volume.

What should be done:
Define a formula to calculate needed size for the Loki volume, based on retention, ingestion rate and number of streams.
Test plan for this could be:

  • ignore the line-size, just measure its average on a real platform and assume all applications will lead to the same average size
  • ignore the retention, just run the benchmarks with a 1 day retention and assume linear scaling (we have a single Loki instance, so that should be roughly true)
  • define a synthetic benchmark, to get data points for multiple (ideally random) combinations of #(streams) x rate(ingestion)
  • from the gathered data, try to build a linear regression model (with OLS)

Implementation proposal (strongly recommended): n/a

Test plan: n/a

@alexandre-allard alexandre-allard added topic:docs Documentation topic:log Anything related to log centralization system labels Jul 29, 2020
@alexandre-allard alexandre-allard added the complexity:medium Something that requires one or few days to fix label Aug 5, 2020
@alexandre-allard alexandre-allard self-assigned this Aug 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
complexity:medium Something that requires one or few days to fix topic:docs Documentation topic:log Anything related to log centralization system
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant