Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LogQL: Add unwrap bytes() conversion function #2876

Merged
merged 2 commits into from
Nov 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/sources/logql/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,9 @@ The unwrap expression is noted `| unwrap label_identifier` where the label ident
Since label values are string, by default a conversion into a float (64bits) will be attempted, in case of failure the `__error__` label is added to the sample.
Optionally the label identifier can be wrapped by a conversion function `| unwrap <function>(label_identifier)`, which will attempt to convert the label value from a specific format.

We currently support only the function `duration_seconds` (or its short equivalent `duration`) which will convert the label value in seconds from the [go duration format](https://golang.org/pkg/time/#ParseDuration) (e.g `5m`, `24s30ms`).
We currently support the functions:
- `duration_seconds(label_identifier)` (or its short equivalent `duration`) which will convert the label value in seconds from the [go duration format](https://golang.org/pkg/time/#ParseDuration) (e.g `5m`, `24s30ms`).
- `bytes(label_identifier)` which will convert the label value to raw bytes applying the bytes unit (e.g. `5 MiB`, `3k`, `1G`).

Supported function for operating over unwrapped ranges are:

Expand Down
1 change: 1 addition & 0 deletions pkg/logql/ast.go
Original file line number Diff line number Diff line change
Expand Up @@ -528,6 +528,7 @@ const (
OpUnwrap = "unwrap"

// conversion Op
OpConvBytes = "bytes"
OpConvDuration = "duration"
OpConvDurationSeconds = "duration_seconds"
)
Expand Down
5 changes: 5 additions & 0 deletions pkg/logql/ast_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,11 @@ func Test_SampleExpr_String(t *testing.T) {
/
count_over_time({namespace="tns"} | logfmt | label_format foo=bar[5m])
)`,
`sum by (job) (
sum_over_time({namespace="tns"} |= "level=error" | json | foo=5 and bar<25ms | unwrap bytes(latency)[5m])
/
count_over_time({namespace="tns"} | logfmt | label_format foo=bar[5m])
)`,
`sum by (job) (
sum_over_time(
{namespace="tns"} |= "level=error" | json | avg=5 and bar<25ms | unwrap duration(latency) [5m]
Expand Down
5 changes: 3 additions & 2 deletions pkg/logql/expr.y
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ import (
%token <val> MATCHERS LABELS EQ RE NRE OPEN_BRACE CLOSE_BRACE OPEN_BRACKET CLOSE_BRACKET COMMA DOT PIPE_MATCH PIPE_EXACT
OPEN_PARENTHESIS CLOSE_PARENTHESIS BY WITHOUT COUNT_OVER_TIME RATE SUM AVG MAX MIN COUNT STDDEV STDVAR BOTTOMK TOPK
BYTES_OVER_TIME BYTES_RATE BOOL JSON REGEXP LOGFMT PIPE LINE_FMT LABEL_FMT UNWRAP AVG_OVER_TIME SUM_OVER_TIME MIN_OVER_TIME
MAX_OVER_TIME STDVAR_OVER_TIME STDDEV_OVER_TIME QUANTILE_OVER_TIME DURATION_CONV DURATION_SECONDS_CONV
MAX_OVER_TIME STDVAR_OVER_TIME STDDEV_OVER_TIME QUANTILE_OVER_TIME BYTES_CONV DURATION_CONV DURATION_SECONDS_CONV

// Operators are listed with increasing precedence.
%left <binOp> OR
Expand Down Expand Up @@ -146,7 +146,8 @@ unwrapExpr:
;

convOp:
DURATION_CONV { $$ = OpConvDuration }
BYTES_CONV { $$ = OpConvBytes }
| DURATION_CONV { $$ = OpConvDuration }
| DURATION_SECONDS_CONV { $$ = OpConvDurationSeconds }
;

Expand Down
Loading