Snmp strip components

[ragzilla]

Adds an oid_strip_index_components option to the snmp input to allow for stripping variable trailing OID sub-indexes (e.g. CISCO-IETF-ISIS-MIB L2 circuits)

Required for all PRs:

• Signed CLA.
• Associated README.md updated.
• Has appropriate unit tests.

[danielnelson]

@ragzilla I don't entirely understand the existing oid_index_suffix option, can you give me an example or how it works and why we we need this new variant?

[ragzilla]

Sure, oid_index_suffix is used when the SNMP agent appends an additional value (usually a sub-index) after the actual indexed value:

$ grep ciiPacketCountLSPs foo.isis 
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.39.level1.sending = Counter32: 0 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.39.level1.receiving = Counter32: 0 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.39.level2.sending = Counter32: 143776 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.39.level2.receiving = Counter32: 1656391 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.40.level1.sending = Counter32: 0 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.40.level1.receiving = Counter32: 0 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.40.level2.sending = Counter32: 865797 frames
CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.40.level2.receiving = Counter32: 1374147 frames

$ snmptranslate -mCISCO-IETF-ISIS-MIB -On CISCO-IETF-ISIS-MIB::ciiPacketCountLSPs.39.level1.sending
.1.3.6.1.4.1.9.10.118.1.5.3.1.6.39.1.1

In this example, we have a fixed MIB determined sub-index. This is the most typical case seen in SNMP agents in the wild. Some vendors in their infinite wisdom occasionally decide to introduce a varying sub-index (usually relating to an index elsewhere in the MIB):

$ grep 'ciiISAdjState' foo.isis   
CISCO-IETF-ISIS-MIB::ciiISAdjState.39.2 = INTEGER: up(3)
CISCO-IETF-ISIS-MIB::ciiISAdjState.40.1 = INTEGER: up(3)

The sub-index in this case isn't useful or worth capturing, but since it varies, it can't be handled with the current oid_index_suffix code path.

Both options have their use cases. The oid_index_suffix could actually have some optimization done to it (since right now to capture multiple sub-indices under a primary SNMP index you have to run multiple parallel gets on the tree).

[danielnelson]

Thanks, that is very helpful. This looks good to me, @phemmer can you also review?

[phemmer]

This smells a bit like the string indexing technique, where the OID suffix is the ASCII representation of a string. E.G. "foobar" is .102.111.111.98.97.114. If this is indeed what is going on here, I think it would be better to declare the length of the index, and discard everything after, as the suffix could be variable length (one OID it might be length 2, another OID it might be length 3).
But given the rarity of this, it's likely not necessary. So I think I could be persuaded to just go with this PR as is.

Edit: Looking at it some more, I think I am leaning towards specifying the index length, instead of the suffix-discard length. In your case, the index length would be 1 (the .39 and .40 values, thus ignoring the .2 and .1).
I'm also willing to bet that the vast majority of users who would end up using this functionality would also end up needing an index length of 1.

[ragzilla]

Similar, in this case it's an additional integer suffix for the ISIS circuit ID (local to the box, and subject to change if the ISIS process restarts so not worth tracking). I'll look at rewriting to a max_index_length (in oid path segments) based approach as that'll cover my use case as well as the ASCII trailer (or other random binary junk) use cases.