promtail dry-run to include fractions of second in timestamp

[samjewell]

Problem

I want to confirm that my timestamp pipeline_stage is working correctly in dry-run mode, before I run promtail "for-real" and send the logs into my Loki instance (because I don't want to sent bad data into Loki).

But dry-run mode only shows whole seconds for the timestamp for each log-line, so I can only confirm that it's working to the nearest second.

Example log lines used for testing this:

level=INFO time=1625253244.5699751 Server=30690 Jitter=4.796ms
level=INFO time=1625253360.99699 Server=34931 Jitter=0.719ms

Pipeline stage:

  pipeline_stages:
  - regex:
      expression: "time=(?P<time>\\S+) "
  - timestamp:
      source: time
      format: Unix
      action_on_failure: skip

Output from promtail -dry-run:

2021-07-02T20:14:04	{filename="/Users/samjewell/fana/run-promtail/example.log", job="testtimestampparsing"}level=INFO time=1625253244.5699751 Server=30690 Jitter=4.796ms
2021-07-02T20:16:00	{filename="/Users/samjewell/fana/run-promtail/example.log", job="testtimestampparsing"}level=INFO time=1625253360.99699 Server=34931 Jitter=0.719ms

(or in colour)

Solution

Possibly switch to tsNs in this output more, or simply add decimals when present?

1625253244569975100 {filename=…

Or

2021-07-02T20:14:04.5699751	{filename=…

Alternatives

Maybe simply a flag which could be enabled, in order to add this additional output to the timestamp, if you don't want to make this change permanent for all users (as it could make the output less well aligned and harder to read in some cases I guess).

Additional context

The sub-second fractions are indeed being handled correctly by promtail and Loki - see:

[dannykopping]

Thanks for submitting this @samjewell!