Missing NetFlow IPFIX Option packets

[SirBreadc]

Use Case

Ref to #15075

Like the above issue can we also get the SNMP option packets showing for IPFIX attached is the telegarf logs with with dump enabled

Expected behavior

Option packets are also writen to output

Actual behavior

Option packets are not written to outputs

Additional info

telegarf_logs.txt

[srebhan]

@SirBreadc unfortunately your log does not contain any IPFIX option packet... I put up a debugging commit in PR #15314 that will output IPFIX OPTIONS FLOWSET FOUND! as a debug message. Could you please the binary in PR #15314, available as soon as CI finished the build, and run the tracing again until you see this message and then attach that log?

[SirBreadc]

#show flow exporter sdwan_flow_exporter_1 
Flow Exporter sdwan_flow_exporter_1:
  Description:              export flow records to collector
  Export protocol:          IPFIX (Version 10)
  Transport Configuration:
    Destination type:       IP
    Destination IP address: <IP>
    VRF label:              1
    Source IP address:      <IP>
    Source Interface:       Loopback0
    Transport Protocol:     UDP
    Destination Port:       2055
    Source Port:            58748
    DSCP:                   0x0
    TTL:                    255
    MTU:                    1280
    Output Features:        Used
  Export template data timeout:        60
  Options Configuration:
    interface-table (timeout 60 seconds)  (active)
    application-table (timeout 60 seconds)  (active)
    application-attributes (timeout 60 seconds)  (active)
    tunnel-tloc-table (timeout 60 seconds)  (active)

Flow Exporter sdwan_flow_exporter_1:
  Client: Option options application-attributes
  Exporter Format: IPFIX (Version 10)
  Template ID    : 256
  Source ID      : 6
  Record Size    : 258
  Template layout
  ____________________________________________________________________________________________________________________________________________
  |                           Field                             | Ent.ID | Field ID |   Full   | Offset |  Size |      App.Id     | SubApp.ID|
  |                                                             |        |          | Field ID |        |       | Eng.ID | Sel.ID |          |
  --------------------------------------------------------------------------------------------------------------------------------------------
  | APPLICATION ID                                              |        |       95 |       95 |      0 |     4 |        |        |          |
  | application category name                                   |      9 |    12232 |    45000 |      4 |    32 |        |        |          |
  | application sub category name                               |      9 |    12233 |    45001 |     36 |    32 |        |        |          |
  | application group name                                      |      9 |    12234 |    45002 |     68 |    32 |        |        |          |
  | application traffic-class                                   |      9 |    12243 |    45011 |    100 |    32 |        |        |          |
  | application business-relevance                              |      9 |    12244 |    45012 |    132 |    32 |        |        |          |
  | p2p technology                                              |        |      288 |      288 |    164 |    10 |        |        |          |
  | tunnel technology                                           |        |      289 |      289 |    174 |    10 |        |        |          |
  | encrypted technology                                        |        |      290 |      290 |    184 |    10 |        |        |          |
  | application set name                                        |      9 |    12231 |    44999 |    194 |    32 |        |        |          |
  | application family name                                     |      9 |    12230 |    44998 |    226 |    32 |        |        |          |
  --------------------------------------------------------------------------------------------------------------------------------------------

  Client: Option options application-name
  Exporter Format: IPFIX (Version 10)
  Template ID    : 257
  Source ID      : 6
  Record Size    : 83
  Template layout
  ____________________________________________________________________________________________________________________________________________
  |                           Field                             | Ent.ID | Field ID |   Full   | Offset |  Size |      App.Id     | SubApp.ID|
  |                                                             |        |          | Field ID |        |       | Eng.ID | Sel.ID |          |
  --------------------------------------------------------------------------------------------------------------------------------------------
  | APPLICATION ID                                              |        |       95 |       95 |      0 |     4 |        |        |          |
  | application name                                            |        |       96 |       96 |      4 |    24 |        |        |          |
  | application description                                     |        |       94 |       94 |     28 |    55 |        |        |          |
  --------------------------------------------------------------------------------------------------------------------------------------------

  Client: Option options interface-table
  Exporter Format: IPFIX (Version 10)
  Template ID    : 258
  Source ID      : 6
  Record Size    : 106
  Template layout
  ____________________________________________________________________________________________________________________________________________
  |                           Field                             | Ent.ID | Field ID |   Full   | Offset |  Size |      App.Id     | SubApp.ID|
  |                                                             |        |          | Field ID |        |       | Eng.ID | Sel.ID |          |
  --------------------------------------------------------------------------------------------------------------------------------------------
  | INTERFACE INPUT SNMP                                        |        |       10 |       10 |      0 |     4 |        |        |          |
  | interface name short                                        |        |       82 |       82 |      4 |    33 |        |        |          |
  | interface name long                                         |        |       83 |       83 |     37 |    65 |        |        |          |
  | interface output snmp                                       |        |       14 |       14 |    102 |     4 |        |        |          |
  --------------------------------------------------------------------------------------------------------------------------------------------

  Client: Option options tunnel-tloc-table
  Exporter Format: IPFIX (Version 10)
  Template ID    : 259
  Source ID      : 6
  Record Size    : 52
  Template layout
  ____________________________________________________________________________________________________________________________________________
  |                           Field                             | Ent.ID | Field ID |   Full   | Offset |  Size |      App.Id     | SubApp.ID|
  |                                                             |        |          | Field ID |        |       | Eng.ID | Sel.ID |          |
  --------------------------------------------------------------------------------------------------------------------------------------------

  | TLOC TABLE OVERLAY SESSION ID                               |      9 |    12435 |    45203 |      0 |     4 |        |        |          |
  | tloc local color                                            |      9 |    12437 |    45205 |      4 |    16 |        |        |          |
  | tloc remote color                                           |      9 |    12439 |    45207 |     20 |    16 |        |        |          |
  | tloc tunnel protocol                                        |      9 |    12440 |    45208 |     36 |     8 |        |        |          |
  | tloc local system ip address                                |      9 |    12436 |    45204 |     44 |     4 |        |        |          |
  | tloc remote system ip address                               |      9 |    12438 |    45206 |     48 |     4 |        |        |          |
  --------------------------------------------------------------------------------------------------------------------------------------------

Would be be able to get all these option packets converted into measurements?
attached is a list of all the option packets which we are sending for IPFIX and a log file too :)
ipfixlogs2.log

[srebhan]

@SirBreadc please test the PR in #15314 and let me know if this work for you!

Would be be able to get all these option packets converted into measurements?

Yes I think so, if the device sends those. In your data there were only application packets and a lot of them, so please be aware that this might produce many metrics...

[SirBreadc]

@srebhan Thanks this is working :) so we see all the snmp option packets making to the netflow_options measurement :)

Yeah we need to tweak our configurations on the box so it sends less frequently.