Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error using json_query multiselect to reduce telegraf JSON data processing #7699

Closed
piyushsharma opened this issue Jun 17, 2020 · 0 comments · Fixed by #7756
Closed

error using json_query multiselect to reduce telegraf JSON data processing #7699

piyushsharma opened this issue Jun 17, 2020 · 0 comments · Fixed by #7756
Labels
area/json json and json_v2 parser/serialiser related bug unexpected problem or unintended behavior
Milestone

Comments

@piyushsharma
Copy link

Relevant telegraf.conf:

[[inputs.tail]]
      files = ["/..../multiselect.log"]
      data_format = "json"
      tag_keys = ["@message","@fields.ouuid"]
      json_query = '{@timestamp,@message,@fields.duration,@fields.responseStatus}'
      json_time_key = "@timestamp"
      json_time_format = "2006-01-02T15:04:05Z07:00"

System info:

Telegraf 1.14.2
macOS Cataline 10.15.5

Steps to reproduce:

  1. Sample Data:
{
  "@message": "Finished Request",
  "@timestamp": "2020-06-09T12:49:18.747Z",
  "@fields": {
    "ouuid": "nav",
    "event": "request",
    "method": "GET",
    "path": "xyz",
    "responseStatus": 200,
    "duration": 10.3845,
    "requestId": "ddd",
    "referer": "abc",
    "level": "info",
    "label": 100
  }
}

This type of JSON is present in each line of the file that I am processing.

  1. Run telegraf --config telegraf.conf

Expected behavior:

See metrics being pushed to local influxdb.

Actual behavior:

I get the following error:

panic: runtime error: index out of range [0] with length 0

goroutine 28 [running]:
github.com/tidwall/gjson.Get(0xc0005000f0, 0xe6, 0xc000052ec1, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/pkg/mod/github.com/tidwall/gjson@v1.3.0/gjson.go:1722 +0xe24
github.com/tidwall/gjson.Get(0xc0005000f0, 0xe6, 0xc000052ec0, 0x3d, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/pkg/mod/github.com/tidwall/gjson@v1.3.0/gjson.go:1734 +0x4ec
github.com/tidwall/gjson.getBytes(0xc0005000f0, 0xe6, 0xf0, 0xc000052ec0, 0x3d, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/pkg/mod/github.com/tidwall/gjson@v1.3.0/gjson_ngae.go:18 +0x10d
github.com/tidwall/gjson.GetBytes(...)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/pkg/mod/github.com/tidwall/gjson@v1.3.0/gjson.go:1814
github.com/influxdata/telegraf/plugins/parsers/json.(*Parser).Parse(0xc000244320, 0xc0005000f0, 0xe6, 0xf0, 0xe6, 0xf0, 0xc0001f81e0, 0xe6, 0xc00048c080)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/src/github.com/influxdata/telegraf/plugins/parsers/json/parser.go:189 +0x41b
github.com/influxdata/telegraf/plugins/inputs/tail.parseLine(0x68a8a80, 0xc000244320, 0xc0001f81e0, 0xe6, 0xc0001f8101, 0xe6, 0x0, 0x0, 0x0, 0x0)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/src/github.com/influxdata/telegraf/plugins/inputs/tail/tail.go:206 +0x280
github.com/influxdata/telegraf/plugins/inputs/tail.(*Tail).receiver(0xc000120200, 0x68a8a80, 0xc000244320, 0xc0002e2bb0)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/src/github.com/influxdata/telegraf/plugins/inputs/tail/tail.go:222 +0x25d
github.com/influxdata/telegraf/plugins/inputs/tail.(*Tail).tailNewFiles.func1(0xc000120200, 0x68a8a80, 0xc000244320, 0xc0002e2bb0)
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/src/github.com/influxdata/telegraf/plugins/inputs/tail/tail.go:177 +0x76
created by github.com/influxdata/telegraf/plugins/inputs/tail.(*Tail).tailNewFiles
	/private/tmp/telegraf-20200429-52913-1ietmxf/telegraf-1.14.2/src/github.com/influxdata/telegraf/plugins/inputs/tail/tail.go:175 +0x530

Additional info:

Some more context and discussion around this issue can be seen in the thread below:
#1363 (comment)

@danielnelson danielnelson added this to the 1.14.5 milestone Jun 17, 2020
@danielnelson danielnelson added area/json json and json_v2 parser/serialiser related bug unexpected problem or unintended behavior labels Jun 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/json json and json_v2 parser/serialiser related bug unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants