Configuration plans for Loki charm

[Abuelodelanada]

Enhancement Proposal

Read this post:

The context is that after instrumenting my machine charm with grafana-agent and setting up a local COS-lite environment, I was able to see logs in Grafana and proceeded to setup a few panels that displayed logs for different sources - syslog, model logs, unit logs, etc. Mainly to observe what the difference was between them. That’s when I ran into a bit of an issue, my Grafana panels all timed out and displayed an error “too many outstanding requests”.

Further investigation landed me on a few threads - like - this suggesting changes to Loki config.

The Loki charm currently doesn’t offer much in the way of config so my question is how will users of the COS-lite stack be able to tune their deployment. My understanding is that sensible charm defaults should get one most of the way there but predictably there might be a need to configure parameters for large and small scale operations.

[Abuelodelanada]

@sed-i said:

Alertmanager has the simplest config yaml of them all and we ended up with providing it in its entirety via config option, rather than deconstructing to reconstruct.

See this config option

Is this possible in Loki? Is there any other option?

[lucabello]

The root issue should be addressed by incrementing the default value of the max_concurrent config; see this comment for reference.