Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GNMI plugin missing measurement name when when writing to influxdb #8783

Closed
jrametta opened this issue Feb 2, 2021 · 8 comments
Closed

GNMI plugin missing measurement name when when writing to influxdb #8783

jrametta opened this issue Feb 2, 2021 · 8 comments
Assignees
Labels
area/gnmi bug unexpected problem or unintended behavior

Comments

@jrametta
Copy link

jrametta commented Feb 2, 2021

Hi, I was not seeing GNMI metric data being sent to influxdb, so I instead enabled the file output plugin with data_format="influx". I am seeing similar error to #6391

telegraf_1    | 2021-02-02T07:01:10Z D! [inputs.gnmi] Connection to gNMI device 10.196.21.52:50051 established                                                                                                                      
telegraf_1    | 2021-02-02T07:01:20Z D! [outputs.file] Could not serialize metric: invalid name                                                                                                                                     
telegraf_1    | 2021-02-02T07:01:20Z D! [outputs.file] Wrote batch of 1 metrics in 166.729µs                                                                                                                                        
telegraf_1    | 2021-02-02T07:01:20Z D! [outputs.file] Buffer fullness: 0 / 10000000 metrics                                                                                                                                        

Relevant telegraf.conf:

  [[inputs.gnmi.subscription]]                                                                                                                                                                                                      
    origin = "device"                                                                                                                                                                                                               
    path = "/System/procsys-items/sysload-items"                                                                                                                                                                                    
    subscription_mode = "sample"                                                                                                                                                                                                    
    sample_interval = "10s"                                                                                                                                                                                                         

If I set file output plugin to json, I can see name is empty in the output ("name":"")

telegraf_1    | 2021-02-02T07:03:53Z D! [inputs.gnmi] Connection to gNMI device 10.196.21.52:50051 established¬¬                                                                                                                    
telegraf_1    | {"fields":{"device:/System/procsys_items_sysload-items_loadAverage15m":"0.5","device:/System/procsys_items_sysload-items_loadAverage1m":"0.14","device:/System/procsys_items_sysload-items_loadAverage5m":"0.38","device:/System/procsys_items_sysload-items_loadAverage5sec":"9.0","device:/System/procsys_items_sysload-items_name":"sysload","device:/System/procsys_items_sysload-items_runProc":1,"device:/System/procsys_items_sysload-items_totalProc":359},"name":"","tags":{"host":"telemetry2.lab","path":"","source":"10.196.21.52"},"timestamp":1612249437}                                                                                                                     

Per the other defect mentioned, if I configure name_override, then the measurement successfully written and given the configured name.

System info:

  • Telegraf 1.17.1 (git: HEAD 8907e61)
  • cisco Nexus9000 93180YC-EX
  • NXOS: version 9.3(5)

Steps to reproduce:

  1. configure GNMI plugin with subscription to Cisco
  2. start telegraf

Expected behavior:

Influxdb measurement is successfully written without error

Actual behavior:

Receive "Could not serialize metric: invalid name" error msg

This error seems to occur for any path given, not just the one mentioned.

@jrametta jrametta added the bug unexpected problem or unintended behavior label Feb 2, 2021
@jrametta
Copy link
Author

jrametta commented Feb 3, 2021

This error is only seen when using json encoding (device capabilities reports both json and proto support)

[[inputs.gnmi]]
    encoding = "json"

After switching to proto, the problem went away. Please close this if it seems like vendor or user error.

@veorlo
Copy link

veorlo commented Mar 26, 2021

I have the same issue. It appears that the output/processor plugins used do not matter. If I dump raw metrics at the processor stage or outputs stage, they both produce an empty string for "name".

Telegraf versions tried: 1.17.3, 1.18.0
Platform returning data to gNMI: Cisco IOS XR

@sjwang90
Copy link
Contributor

sjwang90 commented Apr 9, 2021

@veorlo Does your issue persist as well with just encoding = "proto"?

@tgupta3
Copy link
Contributor

tgupta3 commented Apr 11, 2021

The issue still exists even if we use proto.

@sjwang90
Copy link
Contributor

@sbyx I don't think this is related to #9117 but wanted to see if you had any insight on this issue too.

@veorlo
Copy link

veorlo commented Apr 12, 2021

@sjwang90 I do not experience the issue for the YANG model I'm working with while using encoding = "proto". The YANG model origin is Cisco-IOS-XR-ipv4-bgp-oper found here. My apologies for the lacking info. I will try to add more logs/details when I can.

That said, I've been working with Cisco to determine why some data from that YANG model is missing in the metric fields the gNMI plugin produces while using proto encoding. Using JSON, I get the missing data, but some branches within the specified path are inexplicably joined by underscores and with added list indices (e.g. origin:/some/path/to/my/data would return fields keyed like origin:/some/path/to_my_0_data, where 0 is the array index of a specific data list item). They have investigated and said it must be some decoding/parsing issue within the gNMI plugin. The current solution for that is to use JSON encoding because the missing data does appear at least. For now, I'm working around the field key structure. I will open a separate issue for that if I can gather sufficient evidence that would help in resolving it.

I hope this info can help despite being quite vague :)

@srebhan
Copy link
Member

srebhan commented Nov 23, 2022

@jrametta and @veorlo can you please test a nightly build or the binary in PR #12258? Let me know if it fixes your problem!

@srebhan srebhan self-assigned this Nov 23, 2022
@srebhan
Copy link
Member

srebhan commented Nov 30, 2022

I'm closing this issue as the problem is likely fixed with PR #12258. If the problem still persists, please feel free to reopen the issue or to create a new one.

@srebhan srebhan closed this as completed Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/gnmi bug unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

6 participants