You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
We have promtail configured to forward logs from journald running as a SystemD unit. When promtail encounters an error streaming from journald the service stops forwarding logs until the unit is restarted.
To Reproduce
Run promtail as a SystemD unit streaming and forwarding journald
Encounter error streaming journald (we are working on trying to figure out what might be causing this error here, bad message seems to indicate the stream encountered a non text/string character)
New log entries will not appear in Loki until the promtail SystemD unit is restarted (we typically detect this via loki-canary).
Expected behavior
Promtail continues to stream and forward logs or exits so it can be restarted by SystemD.
Environment:
Infrastructure: SystemD unit on ec2 instances
Screenshots, Promtail config, or terminal output
Log from promtail:
level=error ts=2020-10-26T14:32:39.993094479Z caller=journaltarget.go:174 msg="received error during sdjournal follow" err="failed to read message field: bad message"
Unfortunatelly I can't show the message itself without forking the library
we are using.
Fixesgrafana#2812.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Unfortunatelly I can't show the message itself without forking the library
we are using.
Fixes#2812.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Describe the bug
We have promtail configured to forward logs from journald running as a SystemD unit. When promtail encounters an error streaming from journald the service stops forwarding logs until the unit is restarted.
To Reproduce
Expected behavior
Promtail continues to stream and forward logs or exits so it can be restarted by SystemD.
Environment:
Screenshots, Promtail config, or terminal output
Log from promtail:
Promtail config:
The text was updated successfully, but these errors were encountered: