Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

could not get net class info: error obtaining net class info: open /host/sys/class/net/calixxx #1840

Closed
yogeek opened this issue Sep 7, 2020 · 2 comments · Fixed by #1986

Comments

@yogeek
Copy link

yogeek commented Sep 7, 2020

Host operating system: output of uname -a

Linux ip-10-XX-XXX-XXX.eu-central-1.compute.internal 5.3.0-1023-aws #25~18.04.1-Ubuntu SMP Fri Jun 5 15:18:30 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

quay.io/prometheus/node-exporter:v1.0.1 (daemonset in k8s)

node_exporter command line flags

Args:
      --web.listen-address=127.0.0.1:9100
      --path.procfs=/host/proc
      --path.sysfs=/host/sys
      --path.rootfs=/host/root
      --no-collector.wifi
      --no-collector.hwmon
      --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
      --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

Are you running node_exporter in Docker?

yes, daemonset in k8s (installed with kube-prometheus)

What did you do that produced an error?

nothing... the error seems to always appear on 1 of the k8s node and only one and seems to cause an alert "NodeDown" to be raised (whereas the node is not down)

What did you expect to see?

no error

What did you see instead?

$ kubectl logs -f node-exporter-sgtzd node-exporter
level=info ts=2020-09-04T11:13:17.144Z caller=node_exporter.go:191 msg="Listening on" address=127.0.0.1:9100
level=info ts=2020-09-04T11:13:17.144Z caller=tls_config.go:170 msg="TLS is disabled and it cannot be enabled on the fly." http2=false
level=error ts=2020-09-04T11:26:29.846Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=2.10437765 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/cali2ee73163215: no such file or directory"
level=error ts=2020-09-04T11:49:27.746Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=0.804938467 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/calib6a323c103a: no such file or directory"
level=error ts=2020-09-07T16:33:57.942Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=13.700078944 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/calib499ef8e157: no such file or directory"
level=error ts=2020-09-07T16:36:44.250Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=29.709046202 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/calic59bbe41dbb: no such file or directory"
level=error ts=2020-09-07T16:36:56.143Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=24.587578661 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/calic59bbe41dbb: no such file or directory"
level=error ts=2020-09-07T16:36:57.042Z caller=collector.go:161 msg="collector failed" name=netclass duration_seconds=24.599555658 err="could not get net class info: error obtaining net class info: open /host/sys/class/net/calic59bbe41dbb: no such file or directory"
@discordianfish
Copy link
Member

discordianfish commented Sep 14, 2020

It's probably some sort of permission / dev namespace issue, not an issue in node-exporter itself.

I was about to suggest to just ignore them via --collector.netclass.ignored-devices but that would do the filtering only after the error here: https://github.com/prometheus/procfs/blob/master/sysfs/net_class.go#L120

@SuperQ @pgier We could just ignore field not found errors there as well, wdyt?

SuperQ added a commit that referenced this issue Mar 3, 2021
* Handle no such file and permission denied errors.
* Reduce excessive error wrapping.

Fixes: #1840

Signed-off-by: Ben Kochie <superq@gmail.com>
SuperQ added a commit that referenced this issue Mar 3, 2021
* Handle no such file and permission denied errors.
* Reduce excessive error wrapping.

Fixes: #1840

Signed-off-by: Ben Kochie <superq@gmail.com>
SuperQ added a commit that referenced this issue Mar 5, 2021
* Handle no such file and permission denied errors.
* Reduce excessive error wrapping.

Fixes: #1840

Signed-off-by: Ben Kochie <superq@gmail.com>
@ringyear
Copy link

image
It comes to a situation that error with "failed to read file", did this be silence missing as well?

oblitorum pushed a commit to shatteredsilicon/node_exporter that referenced this issue Apr 9, 2024
* Handle no such file and permission denied errors.
* Reduce excessive error wrapping.

Fixes: prometheus#1840

Signed-off-by: Ben Kochie <superq@gmail.com>
oblitorum pushed a commit to shatteredsilicon/node_exporter that referenced this issue Apr 9, 2024
* Handle no such file and permission denied errors.
* Reduce excessive error wrapping.

Fixes: prometheus#1840

Signed-off-by: Ben Kochie <superq@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants