most variables not exported due to "unsupported value type"

[x3nb63]

i am running the docker image druggeri/nut_exporter:3.1.1@sha256:0d9a0a00554081876178369ab9d46717e002fcf550b18dcd85f98c315438b524 and experience pretty much all variables not being exported, except ups.load, ups.status, battery.charge, battery.voltage and two input.* ones.

See these log lines:

ts=2024-02-14T10:25:54.902Z caller=nut_exporter.go:156 level=info msg="Authenticating to NUT server"
ts=2024-02-14T10:25:54.902Z caller=nut_exporter.go:226 level=info msg="Starting nut_exporter" version=3.1.1
ts=2024-02-14T10:25:54.903Z caller=tls_config.go:238 level=info msg="Listening on" address=[::]:9100
ts=2024-02-14T10:25:54.903Z caller=tls_config.go:241 level=info msg="TLS is disabled." http2=false address=[::]:9100
ts=2024-02-14T10:26:48.444Z caller=nut_exporter.go:126 level=info msg="Creating new registry, handler, and collector for UPS `ks01:3493/ups1`"
ts=2024-02-14T10:26:48.444Z caller=nut_collector.go:302 level=debug msg="Connecting to server and verifying `ups1` is a valid UPS name" server=ks01
ts=2024-02-14T10:26:48.629Z caller=nut_collector.go:326 level=debug msg="UPS name detection" name=ups1
ts=2024-02-14T10:26:48.629Z caller=nut_collector.go:332 level=debug msg="Validity result for UPS named `ups1`" valid=true
ts=2024-02-14T10:26:48.629Z caller=nut_collector.go:85 level=debug msg="Connecting to server" server=ks01 port=3493
ts=2024-02-14T10:26:48.639Z caller=nut_collector.go:97 level=debug msg="Connected to server" server=ks01
ts=2024-02-14T10:26:48.640Z caller=nut_collector.go:102 level=debug msg=Authenticated server=ks01 user=monitor
ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:113 level=debug msg="Instantiated UPS" name=ups1
ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:162 level=debug msg="UPS info" name=ups1 description=Unavailable master=false nmumber_of_logins=4
...
... ==> these are OK:
33-ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:176 level=debug variable_name=battery.charge value=100 type=INTEGER description="Battery charge (percent of full)" writeable=false maximum_length=0 original_type=NUMBER
34:ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
--
47-ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:176 level=debug variable_name=battery.voltage value=26.4 type=FLOAT_64 description="Battery voltage (V)" writeable=false maximum_length=0 original_type=NUMBER
48:ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
49-ts=2024-02-14T10:26:48.760Z caller=nut_collector.go:176 level=debug variable_name=battery.voltage.nominal value=24 type=INTEGER description="Nominal battery voltage (V)" writeable=false maximum_length=0 original_type=NUMBER
50:ts=2024-02-14T10:26:48.761Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
--
89-ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:176 level=debug variable_name=input.voltage value=226 type=FLOAT_64 description="Input voltage (V)" writeable=false maximum_length=0 original_type=NUMBER
90:ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
91-ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:176 level=debug variable_name=input.voltage.nominal value=230 type=INTEGER description="Nominal input voltage (V)" writeable=false maximum_length=0 original_type=NUMBER
92:ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
--
101-ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:176 level=debug variable_name=ups.load value=37 type=INTEGER description="Load on UPS (percent of full)" writeable=false maximum_length=0 original_type=NUMBER
102:ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
--
111-ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:176 level=debug variable_name=ups.status value=OL type=STRING description="UPS status" writeable=false maximum_length=0 original_type=NUMBER
112:ts=2024-02-14T10:26:48.762Z caller=nut_collector.go:192 level=debug msg="Export the variable? true"
...
...
... ==> while these fail:
148-ts=2024-02-14T10:27:48.576Z caller=nut_collector.go:176 level=debug variable_name=battery.charge.low value=10 type=INTEGER description="Remaining battery level when UPS switches to LB (percent)" writeable=true maximum_length=10 original_type=STRING
149:ts=2024-02-14T10:27:48.576Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
150-ts=2024-02-14T10:27:48.576Z caller=nut_collector.go:176 level=debug variable_name=battery.charge.warning value=20 type=INTEGER description="Battery level when UPS switches to Warning state (percent)" writeable=false maximum_length=0 original_type=NUMBER
151:ts=2024-02-14T10:27:48.576Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
...
198-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=input.transfer.high value=295 type=INTEGER description="High voltage transfer point (V)" writeable=true maximum_length=10 original_type=STRING
199:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
200-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=input.transfer.low value=167 type=INTEGER description="Low voltage transfer point (V)" writeable=true maximum_length=10 original_type=STRING
201:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
--
206-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=output.voltage value=226 type=FLOAT_64 description="Output voltage (V)" writeable=false maximum_length=0 original_type=NUMBER
207:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
208-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=ups.beeper.status value=true type=STRING description="UPS beeper status" writeable=false maximum_length=0 original_type=NUMBER
209:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
210-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=ups.delay.shutdown value=20 type=INTEGER description="Interval to wait after shutdown with delay command (seconds)" writeable=true maximum_length=10 original_type=STRING
211:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"
212-ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:176 level=debug variable_name=ups.delay.start value=30 type=INTEGER description="Interval to wait before (re)starting the load (seconds)" writeable=true maximum_length=10 original_type=STRING
213:ts=2024-02-14T10:27:48.578Z caller=nut_collector.go:269 level=debug msg="Export the variable? false" count=7 variables="unsupported value type"

these values are numbers, INTEGER mostly and some it appears to convert from STRING ... but then decides to not export.

Another observation is: those that are exported are missing labels. For example:

network_ups_tools_input_voltage{instance="prometheus-nut-exporter:9100",job="ups"} = 226

Is all that gets scraped. Both labels are from prometheus, none from nur-exporter. Not even the ups name of "ups1" it successfully recognized. And I dont use relabel_configs: so far.

[DRuggeri]

Hey there @x3nb63. Can you share the full command line arguments or query string parameters sent to the exporter? A quick check confirms that you should only see the 'false' for exporting a variable if the argument that limits what to export is set. There is also a default filter applied based on the most common attributes folks seem to be interested in that is probably coming into play.

You are also correct to note that the UPS name isn't exported as a label since the exporter expects there to either be only one UPS or the UPS to be scraped is passed as a parameter (allowing you to rely on the job name or other labels automatically set by prom)

[x3nb63]

hello @DRuggeri ; here is what my prometheus /presumably/ should be doing, done by hand.

No labels except for network_ups_tools_device_info and the _status flags themselves.

/prometheus $ nc -v prometheus-nut-exporter 9100
prometheus-nut-exporter (10.42.2.232:9100) open
GET /ups_metrics?server=ks01&ups=ups1 HTTP/1.0

HTTP/1.0 200 OK
Content-Type: text/plain; version=0.0.4; charset=utf-8
Date: Mon, 19 Feb 2024 15:24:18 GMT

# HELP network_ups_tools_battery_charge Battery charge (percent of full) (battery.charge)
# TYPE network_ups_tools_battery_charge gauge
network_ups_tools_battery_charge 100
# HELP network_ups_tools_battery_voltage Battery voltage (V) (battery.voltage)
# TYPE network_ups_tools_battery_voltage gauge
network_ups_tools_battery_voltage 26.4
# HELP network_ups_tools_battery_voltage_nominal Nominal battery voltage (V) (battery.voltage.nominal)
# TYPE network_ups_tools_battery_voltage_nominal gauge
network_ups_tools_battery_voltage_nominal 24
# HELP network_ups_tools_device_info UPS Device information
# TYPE network_ups_tools_device_info gauge
network_ups_tools_device_info{contact="",description="",location="",macaddr="",mfr="CPS",model="VP1600ELCD",part="",serial="",type="ups"} 1
# HELP network_ups_tools_input_voltage Input voltage (V) (input.voltage)
# TYPE network_ups_tools_input_voltage gauge
network_ups_tools_input_voltage 230
# HELP network_ups_tools_input_voltage_nominal Nominal input voltage (V) (input.voltage.nominal)
# TYPE network_ups_tools_input_voltage_nominal gauge
network_ups_tools_input_voltage_nominal 230
# HELP network_ups_tools_ups_load Load on UPS (percent of full) (ups.load)
# TYPE network_ups_tools_ups_load gauge
network_ups_tools_ups_load 37
# HELP network_ups_tools_ups_status UPS status (ups.status)
# TYPE network_ups_tools_ups_status gauge
network_ups_tools_ups_status{flag="BOOST"} 0
network_ups_tools_ups_status{flag="BYPASS"} 0
network_ups_tools_ups_status{flag="CAL"} 0
network_ups_tools_ups_status{flag="CHRG"} 0
network_ups_tools_ups_status{flag="DISCHRG"} 0
network_ups_tools_ups_status{flag="FSD"} 0
network_ups_tools_ups_status{flag="HB"} 0
network_ups_tools_ups_status{flag="LB"} 0
network_ups_tools_ups_status{flag="OB"} 0
network_ups_tools_ups_status{flag="OFF"} 0
network_ups_tools_ups_status{flag="OL"} 1
network_ups_tools_ups_status{flag="OVER"} 0
network_ups_tools_ups_status{flag="RB"} 0
network_ups_tools_ups_status{flag="SD"} 0
network_ups_tools_ups_status{flag="TRIM"} 0
# HELP promhttp_metric_handler_errors_total Total number of internal errors encountered by the promhttp metric handler.
# TYPE promhttp_metric_handler_errors_total counter
promhttp_metric_handler_errors_total{cause="encoding"} 0
promhttp_metric_handler_errors_total{cause="gathering"} 0
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 7498
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

the variables in the default filter are reported by that ups, all of them:

[root@km1 ~]# upsc ups1@ks01
Init SSL without certificate database
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 20
battery.mfr.date: CPS
battery.runtime: 1260
battery.runtime.low: 300
battery.type: PbAcid
battery.voltage: 26.4
battery.voltage.nominal: 24
device.mfr: CPS
device.model: VP1600ELCD
device.type: ups
driver.name: usbhid-ups
driver.parameter.bus: 002
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.product: VP1600ELCD
driver.parameter.productid: 0501
driver.parameter.synchronous: auto
driver.parameter.vendor: CPS
driver.parameter.vendorid: 0764
driver.version: 2.8.0
driver.version.data: CyberPower HID 0.6
driver.version.internal: 0.47
driver.version.usb: libusb-1.0.23 (API: 0x1000107)
input.transfer.high: 295
input.transfer.low: 167
input.voltage: 230.0
input.voltage.nominal: 230
output.voltage: 230.0
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.load: 37
ups.mfr: CPS
ups.model: VP1600ELCD
ups.productid: 0501
ups.realpower.nominal: 960
ups.status: OL
ups.test.result: No test initiated
ups.timer.shutdown: -60
ups.timer.start: -60
ups.vendorid: 0764
[root@km1 ~]#

as for the last part: then my understanding of the scrape config may be wrong. I derived it from the example but actually want this one job to scrape three UPSes. So it looks this:

- job_name: 'ups'
  metrics_path: /ups_metrics
  static_configs:
       - targets: [ "prometheus-nut-exporter:9100" ]
  params:
      ups: [ "ups1" , "ups2", "ups3"]
      server: [ "ks01", "ks02", "ks03" ]

which results it getting values I cant distinguish in queries

EDIT: note that its actually so that all three upsmon servers know all three UPSes each, only that per server one is primary while two are secondary. My hope was/is to figure this with relabel_configs and/or by promql query ...

[x3nb63]

alright, part if not origin of this issue may be my misunderstanding of who talks to whom in the NUT system: my upsmon services know about all three UPSes, while the upsd on each of the servers knows only the directly connected one ("primary"), I understood just now - and the later is what the exporter talks to ...

... but I dont want to run 3 nut-exporters (one per upsd / physical server) as is done in the second example cause they don't actually run on these servers anyway but $somewhere in the kubernetes cluster ... and that would reserve node resources unnecessarily. The only reason I would want to run more then one exporter would be load on that pod, say of one can not query all UPSes within 1 Minute - but these would then hide behind one kubernetes 'Service` and therefore be invisible to Prometheus.

So my hope was / is that a single exporter can do this, pretty much no matter of how many UPSes I have.

(and to complicate it further: these servers share these UPSes because they have multiple power-supplies each ... so I actually have multiple sets of servers + UPSes, which may form a scrape job, but ideally not ... which is what needs sorting out in the queries, whether its a a failed power supply is involved ... I thought the labels will allow this)

[x3nb63]

all right, for now I handle it by creating three scrape jobs.

Works, but contradicts my understanding of Prometheus, where scrape jobs are the things that discover what there is pretty much all the time. For instance I have just one scrape job per K8s cluster and each discovers all Pods and other objects for the entire cluster.

following that my ideal scrape config for UPSes would be:

- job_name: 'ups'
  metrics_path: /ups_metrics
  static_configs:
       - targets: [ "prometheus-nut-exporter:9100" ]
  params:
      nut-servers: [ "upsd1", "upsd2", ..., "upsdN" ]

where each upsd could know more than one UPS and therefore I would not even want to specify the ups names but expect the exporter to figure them. Don't know if feasible with upsd?

[sleepy-nols]

Hi, Im a bit late to the party but maybe someone will find this usefull in the future.

I also had the issue that nothing exept 3 variables actually got exported. I just set --nut.vars_enable="" like stated in the README, now all variables get exported :)

EDIT:
My solution above somehow stopped working after a while. Issue #52 solved it for me though!