-
Notifications
You must be signed in to change notification settings - Fork 632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NEW] Output logs as JSON #1006
Comments
There was a previous PR in Redis where we discussed some log format:
More information as well
@azuredream I see you opened the old PR to Redis, would you be willing to also contribute your change to Valkey? |
It's very timely for you to open this issue, we talked about this morning as thinking about revamping logs. Can you add a bit more context about what JSON format you want? Either a similar system that outputs in a format that is a bit of a standard. |
Of course you can have any other fields you want in the JSON such as PID , role , etc... the ISO8601 timestamping is particularly important, because TZ Offset , as we have many geographical locations and need to be able to canonize/reconciliate timestamp in a single timezone wherever the logs are consolidated. The ability to also specify the TZ offset in a particular location is also important in case we are asked to log everything in UTC. Also the ability to log to a local file and/or to STDOUT is very important ( in Kubernetes) and thus to be able to choose either or both from the config file of Valkey as an option. Something along these lines |
An example of why being able to get the logs in a structured format (like json or logfmt) would be a good thing, is that it makes it easier to consume and automatically tag log entries in systems like Grafana Loki (which has support for parsing both json and logfmt). It would help a lot if all log entries also was defined with a log level (a list of what for example Grafana Loki recognizes as levels can be seen here). EDIT: A more in depth description of structured logs and their pros can be found at https://github.com/go-logr/logr?tab=readme-ov-file#why-structured-logging EDIT2: For anyone wanting to parse the current format, I built this RE2 regex expression for parsing the log entries with loki and the loki-pipeline-stages.regex support: See full loki pipeline stage code options:
loki-pipeline-stages: |
- regex:
# information about log format found in source code here:
# https://github.com/valkey-io/valkey/blob/bf8183d065f07b40d465cca818a51689036b7488/src/server.c#L114
# levels found at:
# https://github.com/valkey-io/valkey/blob/bf8183d065f07b40d465cca818a51689036b7488/src/server.h#L463
# (NB: LL_NOTHING is only used internally in valkey for disabling
# logging entirely)
expression: '^(?P<pid>\d+):(?P<mode>\S) (?P<ts>\d{1,2} \S{1,3} \d{4} \d{1,2}:\d{1,2}:\d{1,2}\.\d{0,3}) (?P<level>[\.\-\*\#]) (?P<message>.*)$'
- replace:
expression: '(\.)'
source: "level"
replace: "debug"
- replace:
expression: '(-)'
source: "level"
replace: "info"
- replace:
expression: '(\*)'
source: "level"
replace: "info"
- replace:
expression: '(#)'
source: "level"
replace: "warn"
- labels:
level:
- timestamp:
source: "timestamp"
format: "02 Jan 2006 15:04:05.999"
# not setting output to msg, as we would lose pid and mode information, in case that is interesting
|
Could it be worth looking into making the logs "compatible" with OpenTelemetry as part of this effort? I see OpenTelemetry is a CNCF project, which "belongs" to the Linux Foundation. |
Add ability to configure log output format and timestamp format in the logs. This change adds two new configs: * `log-format`: Either legacy or logfmt (See https://brandur.org/logfmt) * `log-timestamp-format`: legacy, iso8601 or milliseconds (since the eppch). Related to #1006. Example: ``` $ ./valkey-server /home/zhaoz12/git/valkey/valkey/valkey.conf pid=109463 role=RDB/AOF timestamp="2024-09-10T20:37:25.738-04:00" level=warning message="WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see jemalloc/jemalloc#1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect." pid=109463 role=RDB/AOF timestamp="2024-09-10T20:37:25.738-04:00" level=notice message="oO0OoO0OoO0Oo Valkey is starting oO0OoO0OoO0Oo" pid=109463 role=RDB/AOF timestamp="2024-09-10T20:37:25.738-04:00" level=notice message="Valkey version=255.255.255, bits=64, commit=affbea5d, modified=1, pid=109463, just started" pid=109463 role=RDB/AOF timestamp="2024-09-10T20:37:25.738-04:00" level=notice message="Configuration loaded" pid=109463 role=master timestamp="2024-09-10T20:37:25.738-04:00" level=notice message="monotonic clock: POSIX clock_gettime" pid=109463 role=master timestamp="2024-09-10T20:37:25.739-04:00" level=warning message="Failed to write PID file: Permission denied" ``` --------- Signed-off-by: azuredream <zhaozixuan67@gmail.com>
Would it be possible to get Redis logs output in JSON format with a timestamp that has timezone information as the first field.
This is useful when integrating and using redis in large kubernetes based systems which ingest and demand JSON logs to stdout for all apps.
The text was updated successfully, but these errors were encountered: