SNMP exporter returns only one of both xdsl2ChStatusUnit

[micharambou]

Host operating system: output of uname -a

[alarm@alarm generator]$ docker exec -it snmp-exporter-test uname -a
Linux c443a1a4e72c 6.0.0-rc5-1-aarch64-rc-ARCH #1 SMP PREEMPT Tue Sep 13 12:13:42 MDT 2022 aarch64 GNU/Linux

snmp_exporter version: output of snmp_exporter -version

version=0.24.1

What device/snmpwalk OID are you using?

• 1.3.6.1.2.1.10.251.1.2.2.1.1 # xdsl2ChStatusUnit
• 1.3.6.1.2.1.10.251.1.2.2.1.2 # xdsl2ChStatusActDataRate

MIBs in use:

[alarm@alarm generator]$ ls ~/.snmp/mibs/
IF-MIB  SNMP-FRAMEWORK-MIB  SNMPv2-CONF.txt  SNMPv2-SMI  SNMPv2-TC  VDSL2-LINE-MIB.mib  VDSL2-LINE-TC-MIB.txt

What did you do that produced an error?

Given the following results of snmpwalk cli command targeting a DrayTek Vigor 165 CPE:

[alarm@alarm generator]$ snmpwalk  -On -v 2c  -c public 192.168.1.1 1.3.6.1.2.1.10.251.1.2.2.1.1
.1.3.6.1.2.1.10.251.1.2.2.1.1.4.1 = INTEGER: 1
.1.3.6.1.2.1.10.251.1.2.2.1.1.4.2 = INTEGER: 2

[alarm@alarm generator]$ snmpwalk -On -v 2c  -c public 192.168.1.1 1.3.6.1.2.1.10.251.1.2.2.1.2
.1.3.6.1.2.1.10.251.1.2.2.1.2.4.1 = Gauge32: 107799000
.1.3.6.1.2.1.10.251.1.2.2.1.2.4.2 = Gauge32: 26999000

generator.yml:

auths:
  public_v2:
    version: 2

modules:
  vigor165:
    walk: 
      #VDSL2
      - 1.3.6.1.2.1.10.251.1.2.2.1.1 #	xdsl2ChStatusUnit
      - 1.3.6.1.2.1.10.251.1.2.2.1.2 #	xdsl2ChStatusActDataRate

./generator generate -g vigor_gen_test.yml -o snmp_vigor_test.yml

results in snmp.yml

# WARNING: This file was auto-generated using snmp_exporter generator, manual changes will be lost.
auths:
  public_v2:
    community: public
    security_level: noAuthNoPriv
    auth_protocol: MD5
    priv_protocol: DES
    version: 2
modules:
  vigor165:
    walk:
    - 1.3.6.1.2.1.10.251.1.2.2.1.1
    - 1.3.6.1.2.1.10.251.1.2.2.1.2
    metrics:
    - name: xdsl2ChStatusUnit
      oid: 1.3.6.1.2.1.10.251.1.2.2.1.1
      type: gauge
      help: The termination unit. - 1.3.6.1.2.1.10.251.1.2.2.1.1
      indexes:
      - labelname: ifIndex
        type: gauge
      - labelname: xdsl2ChStatusUnit
        type: gauge
        enum_values:
          1: xtuc
          2: xtur
      enum_values:
        1: xtuc
        2: xtur
    - name: xdsl2ChStatusActDataRate
      oid: 1.3.6.1.2.1.10.251.1.2.2.1.2
      type: gauge
      help: The actual net data rate at which the bearer channel is operating, if
        in L0 power management state - 1.3.6.1.2.1.10.251.1.2.2.1.2
      indexes:
      - labelname: ifIndex
        type: gauge
      - labelname: xdsl2ChStatusUnit
        type: gauge
        enum_values:
          1: xtuc
          2: xtur

What did you expect to see?

SNMP Exporter should return both the xdsl2ChStatusUnit for xtuc and xtur and the corresponding xdsl2ChStatusActDataRate.

What did you see instead?

Only the second of both xdsl2ChStatusUnit and xdsl2ChStatusActDataRate is returned

[I] michael@manjaro-xps13 ~ [124]> curl "http://192.168.100.74:9116/snmp?target=192.168.1.1&auth=public_v2&module=vigor165"
# HELP snmp_scrape_duration_seconds Total SNMP time scrape took (walk and processing).
# TYPE snmp_scrape_duration_seconds gauge
snmp_scrape_duration_seconds{module="vigor165"} 0.010288329
# HELP snmp_scrape_packets_retried Packets retried for get, bulkget, and walk.
# TYPE snmp_scrape_packets_retried gauge
snmp_scrape_packets_retried{module="vigor165"} 0
# HELP snmp_scrape_packets_sent Packets sent for get, bulkget, and walk; including retries.
# TYPE snmp_scrape_packets_sent gauge
snmp_scrape_packets_sent{module="vigor165"} 2
# HELP snmp_scrape_pdus_returned PDUs returned from get, bulkget, and walk.
# TYPE snmp_scrape_pdus_returned gauge
snmp_scrape_pdus_returned{module="vigor165"} 2
# HELP snmp_scrape_walk_duration_seconds Time SNMP walk/bulkwalk took.
# TYPE snmp_scrape_walk_duration_seconds gauge
snmp_scrape_walk_duration_seconds{module="vigor165"} 0.010185404
# HELP xdsl2ChStatusActDataRate The actual net data rate at which the bearer channel is operating, if in L0 power management state - 1.3.6.1.2.1.10.251.1.2.2.1.2
# TYPE xdsl2ChStatusActDataRate gauge
xdsl2ChStatusActDataRate{ifIndex="4",xdsl2ChStatusUnit="2"} 2.6999e+07  <<<<<<
# HELP xdsl2ChStatusUnit The termination unit. - 1.3.6.1.2.1.10.251.1.2.2.1.1
# TYPE xdsl2ChStatusUnit gauge
xdsl2ChStatusUnit{ifIndex="4",xdsl2ChStatusUnit="2"} 2 <<<<<<<

Is this a bug or am i missing something regarding the generator configuration?

[SuperQ]

That does seem like a bug of some kind. Can you post a with a curl when the exporter is running with --log.level=debug?

It may also be helpful to have a tcpdump/wireshark capture of the exporter's SNMP traffic as well as the snmpwalk to see if there are any protocol differences.

Sadly, I have not added support for the Vigor 165 to the draytek_exporter. I think that model uses the old v4 firmware, not the new v5 firmware.

[micharambou]

Thanks for your response.

Here is the debug log output. pcap attached
snmp.tar.gz

[alarm@alarm snmp_exporter]$ docker run --entrypoint "bin/snmp_exporter" -v /home/alarm/docker/snmp_exporter/generator/snmp_vigor_test.yml:/etc/snmp_exporter/snmp.yml -p9116:9116 --name snmp-exporter-test prom/snmp-exporter --log.level="debug" --config.file="/etc/snmp_exporter/snmp.yml"
ts=2023-11-29T19:19:23.464Z caller=main.go:194 level=info msg="Starting snmp_exporter" version="(version=0.24.1, branch=HEAD, revision=f8826bcfadf9d93df9516c660fdc1d503f4e0b95)" concurrency=1
ts=2023-11-29T19:19:23.464Z caller=main.go:195 level=info build_context="(go=go1.21.0, platform=linux/arm64, user=root@8cd478405a6a, date=20230901-07:33:32, tags=netgo)"
ts=2023-11-29T19:19:23.467Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9116
ts=2023-11-29T19:19:23.467Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9116
ts=2023-11-29T19:19:25.721Z caller=collector.go:460 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Starting scrape"
ts=2023-11-29T19:19:25.721Z caller=collector.go:214 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Walking subtree" oid=1.3.6.1.2.1.10.251.1.2.2.1.1
ts=2023-11-29T19:19:25.728Z caller=collector.go:228 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Walk of subtree completed" oid=1.3.6.1.2.1.10.251.1.2.2.1.1 duration_seconds=6.288671ms
ts=2023-11-29T19:19:25.728Z caller=collector.go:214 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Walking subtree" oid=1.3.6.1.2.1.10.251.1.2.2.1.2
ts=2023-11-29T19:19:25.733Z caller=collector.go:228 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Walk of subtree completed" oid=1.3.6.1.2.1.10.251.1.2.2.1.2 duration_seconds=5.435326ms
ts=2023-11-29T19:19:25.734Z caller=collector.go:464 level=debug auth=public_v2 target=192.168.1.1 module=vigor165 msg="Finished scrape" duration_seconds=0.012626934

Some notes that might be of interest:

• Vigor CPE is running on FW 4.2.5_STD

DrayTek> sys version
Router Model: Vigor165    Version: 4.2.5_STD English
Profile version: 3.0.0    Status: 1 (0x12872e04)
Router IP: 192.168.1.1    Netmask: 255.255.255.0
Firmware Build Date/Time: May 12 2023 13:54:44
Router Name: DrayTek
Revision: 16746_797_bf5daf4c5 V421_165166_423FW
Current VDSL2 Firmware Version: 08-0B-02-06-00-07
ADSL Firmware Version: 08-0B-00-0B-00-02 Annex B
VDSL2 Firmware Version: 08-0B-02-06-00-07
Router serial no: None

• I have been able to workaround the issue with this setup

generator.yml

auths:
  public_v2:
    version: 2

modules:
  vigor165:
    walk: 
      #VDSL2
      - 1.3.6.1.2.1.10.251.1.2.2.1

Had to remove the following sections in generated snmp.yml to make it work though (CPE is running in VDSL2 17A mode)

• name: xdsl2ChStatusAtmStatus
• name: xdsl2ChStatusPtmStatus

[micharambou]

pcap of snmpwalk attached
snmpwalk.tar.gz

[alarm@alarm generator]$ snmpwalk  -On -v 2c  -c public 192.168.1.1 1.3.6.1.2.1.10.251.1.2.2.1.1
.1.3.6.1.2.1.10.251.1.2.2.1.1.4.1 = INTEGER: 1
.1.3.6.1.2.1.10.251.1.2.2.1.1.4.2 = INTEGER: 2
[alarm@alarm generator]$ snmpwalk -On -v 2c  -c public 192.168.1.1 1.3.6.1.2.1.10.251.1.2.2.1.2
.1.3.6.1.2.1.10.251.1.2.2.1.2.4.1 = Gauge32: 99998000
.1.3.6.1.2.1.10.251.1.2.2.1.2.4.2 = Gauge32: 26999000

[jjdmol]

We might be experiencing the same issue. In our case, a workaround consisted of creating an additional module in snmp-exporter that only walks the missing OIDs by explicitly specifying their specific instances. It was not sufficient to add those OIDs to our existing module, as snmp-exporter's generator will optimise them away if the tree above it is also walked in the same module.

This behaviour seems to stem from gosnmp, which in some cases does not show all OIDs where snmpwalk does, f.e.:

~/gosnmp$ snmpwalk -c public -v 2c 10.99.250.70 1.3.6.1.4.1.5961.5.6.2.1.2
iso.3.6.1.4.1.5961.5.6.2.1.2.0 = INTEGER: 1
iso.3.6.1.4.1.5961.5.6.2.1.2.1 = INTEGER: 2
iso.3.6.1.4.1.5961.5.6.2.1.2.2 = INTEGER: 3
iso.3.6.1.4.1.5961.5.6.2.1.2.3 = INTEGER: 4
~/gosnmp$ go run examples/walkexample/main.go 10.99.250.70 1.3.6.1.4.1.5961.5.6.2.1.2
.1.3.6.1.4.1.5961.5.6.2.1.2.0 = TYPE 2: 1
.1.3.6.1.4.1.5961.5.6.2.1.2.1 = TYPE 2: 2
.1.3.6.1.4.1.5961.5.6.2.1.2.2 = TYPE 2: 3

Yet gosnmp is able to retrieve the specific instances just fine, if explicitly asked to do so:

~/gosnmp$ go run examples/walkexample/main.go 10.99.250.70 1.3.6.1.4.1.5961.5.6.2.1.2.3
.1.3.6.1.4.1.5961.5.6.2.1.2.3.0 = TYPE 2: 4

Edit: I reported the above issue in gosnmp at gosnmp/gosnmp#467.