Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

udev mapping rule doesn't allow container to start #14

Closed
FuzzyMistborn opened this issue May 11, 2021 · 29 comments
Closed

udev mapping rule doesn't allow container to start #14

FuzzyMistborn opened this issue May 11, 2021 · 29 comments

Comments

@FuzzyMistborn
Copy link

Not sure if this is an RTL_433 issue or a docker issue, but to get around the issue where the /dev/bus/usb path changes I'm trying to create a udev rule to map the device id to /dev/rtl433 and then pass that through to the container. However, the container doesn't seem to like that as I get this error:

rtl_433 version 20.11-117-g445ddda5 branch master at 202104251151 inputs file rtl_tcp RTL-SDR,
Use -h for usage help and see https://triq.org/ for documentation.,
Trying conf file at "rtl_433.conf"...,
Trying conf file at "/root/.config/rtl_433/rtl_433.conf"...,
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...,
Trying conf file at "/etc/rtl_433/rtl_433.conf"...,
Publishing MQTT data to XXXXXXXXX port 1883,
Publishing device info to MQTT topic "rtl_433[/model][/id]".,
Registered 2 out of 186 device decoding protocols [ 40-41 ],
No supported devices found.,

I suspect it's because RTL_433 is set to look for something on the /dev/bus/usb path. When I try to map /dev/rtl433 to something like.... /dev/bus/usb/001/004 I get this instead:

rtl_433 version 20.11-117-g445ddda5 branch master at 202104251151 inputs file rtl_tcp RTL-SDR,
Use -h for usage help and see https://triq.org/ for documentation.,
Trying conf file at "rtl_433.conf"...,
Trying conf file at "/root/.config/rtl_433/rtl_433.conf"...,
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...,
Trying conf file at "/etc/rtl_433/rtl_433.conf"...,
Publishing MQTT data to 192.168.30.12 port 1883,
Publishing device info to MQTT topic "rtl_433[/model][/id]".,
Registered 2 out of 186 device decoding protocols [ 40-41 ],
usb_open error -4,

If I map it to the match USB path of /dev/bus/usb it works. Which kinda defeats the point of a udev rule. Is there a way around this?

@hertzg
Copy link
Owner

hertzg commented May 13, 2021

I would ask about how rtl_433 and the driver that you use (I'm assuming rtl-sdr as you have not specified the command, only output) at https://github.com/merbanan/rtl_433.

But trying to be helpful here, Have you tried using the -d option to specify the input device forrtl_433 and the matching --device option for docker run to expose that device inside the container?
Both of them should give you the flexibility you need and use the manualy mapped path.

In the second example, the usb_open error -4, could be libusb complaining about wrong udev rule maybe or something has claimed it already? 🤷‍♂️ Hard to say from these outputs.

LIBUSB_ERROR_NO_DEVICE | No such device (it may have been disconnected)

Quotes from https://github.com/merbanan/rtl_433#running

[-d <RTL-SDR USB device index> | :<RTL-SDR USB device serial> | <SoapySDR device query> | rtl_tcp | help]

and

		= Input device selection =
	RTL-SDR device driver is available.
  [-d <RTL-SDR USB device index>] (default: 0)
  [-d :<RTL-SDR USB device serial (can be set with rtl_eeprom -s)>]
	To set gain for RTL-SDR use -g <gain> to set an overall gain in dB.
	SoapySDR device driver is available.
  [-d ""] Open default SoapySDR device
  [-d driver=rtlsdr] Open e.g. specific SoapySDR device
	To set gain for SoapySDR use -g ELEM=val,ELEM=val,... e.g. -g LNA=20,TIA=8,PGA=2 (for LimeSDR).
  [-d rtl_tcp[:[//]host[:port]] (default: localhost:1234)
	Specify host/port to connect to with e.g. -d rtl_tcp:127.0.0.1:1234

@hertzg hertzg added the question Further information is requested label May 13, 2021
@FuzzyMistborn
Copy link
Author

I appreciate the help. Again I wasn't sure if this was rtl-433 related or the container. Figured since it's going to be a limitation on the container (ie having to change what device is passed through) I'd raise it here but I have no issue going over to rtl-433 if that's the right spot to go.

Correct I'm running rtl-sdr, here's my docker-compose snippet:

  rtl433:
    image: hertzg/rtl_433:latest
    container_name: rtl433
    devices:
      - /dev/bus/usb/001/006
    command:
      - -Fmqtt://192.168.XXXX:1883,user=XXXX,pass=XXXXXX,retain=1,devices=rtl_433[/model][/id]
    restart: always

I did try to play with the -d flag but even trying to use the serial number -d :000001 doesn't work.

@hertzg
Copy link
Owner

hertzg commented May 14, 2021

There's also --device-cgroup-rule for docker that could help you, seems to be pretty recent addition. I'm going to close this for now as I don't see this as an issue with this repository but feel free to continue with your findings, I'm curious how you will solve this.

A quick search turned some interesting leads that I would investigate:

@hertzg hertzg closed this as completed May 14, 2021
@FuzzyMistborn
Copy link
Author

Thank you for those links. I tried them and still had no luck. Going to give up for now as I don't restart all that often so hopefully it isn't a bit issue. I'll keep trying though.

@hertzg
Copy link
Owner

hertzg commented May 15, 2021

I currently have it setup so that the sdr device is always assigned /dev/bus/usb/001/004 (at boot) on the PI, the rtl_433 serves as a "bridge" between 433Mhz radio waves and MQTT.

@FuzzyMistborn
Copy link
Author

FuzzyMistborn commented May 15, 2021

How did you do that? udev rule mapping it to that path? All I'm looking for is consistency between reboots, the specific path doesn't matter to me.

@hertzg
Copy link
Owner

hertzg commented May 15, 2021

The more I dig into this topic the more I realise I'm lucky 😅. In my case I just have the sdr device connected to the Pi4 port (don't remember which one exactly), I do not restart it at all and it seems to have not been renumerated for the past year or so.

How did you do that?

Nothing, I just not touch the PI at all, I know it's not a solution but so far (about a year) It has kept its id.

The doc mentions that

Stable identifiers are available, for user mode applications that want to use them.

But unfortunately I do not know where and how to access them.

Quote from Linux usbfs docs:

What files are in "devtmpfs"?
Conventionally mounted at /dev/bus/usb/, usbfs features include:

/dev/bus/usb/BBB/DDD ... magic files exposing the each device's configuration descriptors, and supporting a series of ioctls for making device requests, including I/O to devices. (Purely for access by programs.)
Each bus is given a number (BBB) based on when it was enumerated; within each bus, each device is given a similar number (DDD). Those BBB/DDD paths are not "stable" identifiers; expect them to change even if you always leave the devices plugged in to the same hub port. Don't even think of saving these in application configuration files. Stable identifiers are available, for user mode applications that want to use them. HID and networking devices expose these stable IDs, so that for example you can be sure that you told the right UPS to power down its second server. Pleast note that it doesn't (yet) expose those IDs.

Source: https://github.com/torvalds/linux/blob/ef1244124349fea36e4a7e260ecaf156b6b6b22a/Documentation/driver-api/usb/usb.rst#what-files-are-in-devtmpfs

@FuzzyMistborn
Copy link
Author

Sorry I haven't had time to keep working on this. Interestingly I rebooted my server by accident this morning and everything came back up fine. So it may not be as big an issue as I thought.

@CharlesGodwin
Copy link

@hertzg @FuzzyMistborn
I know you closed this but I too have the problem of my radio device wandering between reboots. I had another implementation of RTL_433 at a different location that didn't have this problem, so I checked.

I have found that this works. It seems rtl_433 works it out

   devices:
      - /dev/bus/usb

@FuzzyMistborn
Copy link
Author

Interesting. That definitely would work. My other issue that's more particular to me is that I run things inside linux containers (LXC) so I have to pass through devices to the container. I'd prefer not to pass every USB device through for security reasons. But that's a clever solution.

@CharlesGodwin
Copy link

CharlesGodwin commented May 29, 2021 via email

@hertzg
Copy link
Owner

hertzg commented May 30, 2021

Following up on the quote from kernel about "stable" ids. I was able to only find usb_make_path method from Linux-USB Host Side API, not sure if there is a userspace program that makes use if that api.

@CharlesGodwin
Copy link

I have read an article that discusses how to do it for USB serial devices. I use it for a RS485 dongle but it doesn't seem to work for this application. Want to read it, I can dig it up

@damusmf
Copy link

damusmf commented Jun 25, 2021

i know this is closed, but @FuzzyMistborn i was able to get udev rules working.

File called: /etc/udev/rules.d/99-rtl433.rules
In the file i have one line

SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2838", SYMLINK+="rtl433"

your vendor and product might be different, just use udevadm info --name=/dev/bus/usb/001/003 --attribute-walk but your path might be different, can use lsusb to find the right path

Then in my docker compose i have
devices:
- /dev/rtl433:/dev/bus/usb/001/003

just mapping my static path to a normal /bus/usb path within the container and it was able to find it no problem.

@CharlesGodwin
Copy link

I tried it and didn't work. I get this in my log usb_open error -4

$ cat /etc/udev/rules.d/99-rtl433.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2838", SYMLINK+="rtl433"
$ lsusb
Bus 002 Device 002: ID 0951:1666 Kingston Technology DataTraveler 100 G3/G4/SE9 G2
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 004: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
Bus 001 Device 003: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls -l /dev/rtl433
lrwxrwxrwx 1 root root 15 Jun 25 10:16 /dev/rtl433 -> bus/usb/001/005

docker-compose.yml has

     devices:
        - /dev/rtl433:/dev/bus/usb/001/003

@damusmf
Copy link

damusmf commented Jun 25, 2021

I tried it and didn't work. I get this in my log usb_open error -4

$ cat /etc/udev/rules.d/99-rtl433.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2838", SYMLINK+="rtl433"
$ lsusb
Bus 002 Device 002: ID 0951:1666 Kingston Technology DataTraveler 100 G3/G4/SE9 G2
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 004: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
Bus 001 Device 003: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls -l /dev/rtl433
lrwxrwxrwx 1 root root 15 Jun 25 10:16 /dev/rtl433 -> bus/usb/001/005

docker-compose.yml has

     devices:
        - /dev/rtl433:/dev/bus/usb/001/003

once you make the rule file did you reload them?
udevadm control --reload-rules && udevadm trigger

Edit: and also, make sure you rebuild the container, not just restart it. if none of that works for you then i'm not too sure, it is working for me here.

@damusmf
Copy link

damusmf commented Jun 25, 2021

$ cat /etc/udev/rules.d/99-rtl433.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2838", SYMLINK+="rtl433"
$ lsusb
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 010: ID 0658:0200 Sigma Designs, Inc.
Bus 003 Device 009: ID 1cf1:0030 Dresden Elektronik
Bus 003 Device 012: ID 1a86:e024 QinHeng Electronics
Bus 003 Device 002: ID 1a40:0101 Terminus Technology Inc. Hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls -l /dev/rtl433
lrwxrwxrwx 1 root root 15 Jun 24 18:04 /dev/rtl433 -> bus/usb/001/003

And my docker-compose

  rtl433:
    container_name: rtl433
    image: hertzg/rtl_433:latest
    environment:
      - TZ=America/Chicago
    devices:
      - '/dev/rtl433:/dev/bus/usb/001/003'
    command:
      - '-Fmqtt://10.5.1.122:1883,retain=0'

And logs upon starting container

rtl_433 version 21.05-5-g8263b6c6 branch master at 202105261104 inputs file rtl_tcp RTL-SDR,
Use -h for usage help and see https://triq.org/ for documentation.,
Trying conf file at "rtl_433.conf"...,
Trying conf file at "/root/.config/rtl_433/rtl_433.conf"...,
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...,
Trying conf file at "/etc/rtl_433/rtl_433.conf"...,
Publishing MQTT data to 10.5.1.122 port 1883,
Publishing device info to MQTT topic "rtl_433/ff83f296f1ce/devices[/type][/model][/subtype][/channel][/id]".,
Publishing events info to MQTT topic "rtl_433/ff83f296f1ce/events".,
Publishing states info to MQTT topic "rtl_433/ff83f296f1ce/states".,
Registered 157 out of 186 device decoding protocols [ 1-4 8 11-12 15-17 19-23 25-26 29-36 38-60 63 67-71 73-100 102-105 108-116 119 121 124-128 130-149 151-161 163-168 170-175 177-186 ],
Detached kernel driver,
Found Rafael Micro R820T tuner,
Exact sample rate is: 250000.000414 Hz,
[R82XX] PLL not locked!,
Sample rate set to 250000 S/s.,
Tuner gain set to Auto.,
Tuned to 433.920MHz.,
Allocating 15 zero-copy buffers,
MQTT Connected...,
baseband_demod_FM: low pass filter for 250000 Hz at cutoff 25000 Hz, 40.0 us,
MQTT Connection established.,

@hertzg hertzg added has-workaround and removed question Further information is requested labels Jul 31, 2021
@hertzg
Copy link
Owner

hertzg commented Dec 8, 2021

My current situation and my solution to this issue:

In my case I have two of the same type of devices attached to a host that is running docker, and I use docker-compose.yml to run everything.

version: '3.8'

services:
  rtl_433_587:
    image: hertzg/rtl_433:alpine-3.14-21.05
    devices:
      - /dev/bus/usb
    container_name: rtl_433_587
    hostname: rtl_433_587
    restart: unless-stopped
    command:
      - '-d:${RTL_SERIAL_433_587}'
      - '-f433.587MHz'
      - '-Mtime:unix:usec:utc'
      - '-Mbits'
      - '-Mlevel'
      - '-Mprotocol'
      - '-Mstats:2:300'
      - '-Fmqtt://${MQTT_HOST},user=${MQTT_USER},pass=${MQTT_PASS}'
      - '-Finflux://${INFLUX_HOST}/api/v2/write?org=${INFLUX_ORG}&bucket=${INFLUX_BUCKET},token=${INFLUX_TOKEN}'

  rtl_433_920:
    image: hertzg/rtl_433:alpine-3.14-21.05
    devices:
      - /dev/bus/usb
    container_name: rtl_433_920
    hostname: rtl_433_920
    restart: unless-stopped
    command:
      - '-d:${RTL_SERIAL_433_92}'
      - '-f433.92MHz'
      - '-Mtime:unix:usec:utc'
      - '-Mbits'
      - '-Mlevel'
      - '-Mprotocol'
      - '-Mstats:2:300'
      - '-Fmqtt://${MQTT_HOST},user=${MQTT_USER},pass=${MQTT_PASS}'
      - '-Finflux://${INFLUX_HOST}/api/v2/write?org=${INFLUX_ORG}&bucket=${INFLUX_BUCKET},token=${INFLUX_TOKEN}'

In my case the devices might be mounted with any bus/id pair and I let rtl_433 do the selection of the correct one using it's serial number.

The only downside is that I am exposing the whole /dev/bus/usb to the container but I have this host (VM) dedicated only for rtl_433

root@rtl433 [ ~ ]# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 006: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 005: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
Bus 001 Device 004: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
Bus 001 Device 003: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

You can see an alternatate solution with a startup contributed by a user: #45

@viperk1
Copy link

viperk1 commented Dec 25, 2021

I tried it and didn't work. I get this in my log usb_open error -4

$ cat /etc/udev/rules.d/99-rtl433.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2838", SYMLINK+="rtl433"
$ lsusb
Bus 002 Device 002: ID 0951:1666 Kingston Technology DataTraveler 100 G3/G4/SE9 G2
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 004: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
Bus 001 Device 003: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls -l /dev/rtl433
lrwxrwxrwx 1 root root 15 Jun 25 10:16 /dev/rtl433 -> bus/usb/001/005

docker-compose.yml has

     devices:
        - /dev/rtl433:/dev/bus/usb/001/003

Sorry to revive this, but I had the same error as you. Your docker-compose mapping uses usb/001/003 and your physical device is attached at usb/001/005. If you change the mapping to use 005 instead, then it works. This doesn't really make sense to me because now even with udev rules it's still dependent on what the host device path is, which kinda defeats the purpose.

In my case, my device was on usb/003/008 and I had to use that exact path in the docker device mapping otherwise I'd get the error -4.

@hertzg
Copy link
Owner

hertzg commented Jan 22, 2022

@viperk1 There are few easy options without digging too much into details:

  • Find the device id that kernel assigns on startup, run the container with it and never touch that thing, if something goes wrong restart the host.
  • Expose all (--device /dev/usb) and let rtl_433 pick based on the serial number (using -d :serial_no_goes_here_after_colon)
  • Use a script to find current bus path by vendor id, use it as value for --device and run the container (have a look at the script from @elmicha in Shell script to start the container with the correct bus/device #45)

All of them have different ups and downs but are viable options, depends on your preference / limitations / level of paranioa.

@gwisp2
Copy link

gwisp2 commented Apr 6, 2022

I found a solution that allows to expose only the needed device.

rtl433 requires the device to be accessible via the /dev/bus/usb/XXX/YYY path, however it is possible to create a symlink from this path to the static path while running in a container, so that we don't need any scripts that change docker-compose.yml before actually calling docker-compose.

REAL_PATH="/dev/$(udevadm info --name=/dev/rtl-sdr-dongle -q name)"
mkdir -p $(dirname "$REAL_PATH")
ln -s /dev/rtl-sdr-dongle "$REAL_PATH"

We need udevadm for that to be installed though (package eudev in alpine), but possibly there is a way to do the same without.

@gwisp2
Copy link

gwisp2 commented Apr 6, 2022

I suggest adding the entrypoint script with a workaround.
Something like this.

@JB09
Copy link

JB09 commented Jul 30, 2022

Would be nice if a more proper solution could be found for this. I believe it has something to do with the host passing meta info to the container. If you lsusb inside the container, it reads out the info from the host. Not the device being bound. I tried adding MODE="0666" and GROUP="dialout", but the error -4 remained leading me to believe it's not a permission issue, but rather a derivatives of how program detects the available usb devices.

@gwisp2
Copy link

gwisp2 commented Jul 30, 2022

but rather a derivatives of how program detects the available usb devices

In April I investigated this issue by trying to understand device detection code using strace and libusb source. As I remember libusb can't be asked to use particular udev device path. And you are right that it is device list detection issue.

@JB09
Copy link

JB09 commented Jul 30, 2022

I suggest adding the entrypoint script with a workaround.
Something like this.

@gwisp2 i was able to get this working, thank you for your efforts! I also learned about Dockerfiles as an added bonus!

@JB09
Copy link

JB09 commented Jul 31, 2022

Edit: just wanted to add that it doesn’t appear that command arguments via the compose file can be passed with this method. Based on some googling, maybe due to using the shell form of Entrypoint? Anyways, I was able to get things working using a .conf file (with my MQTT server) if anyone else has the same issue.

Alternatively, maybe these could be set via env variables?

@gwisp2
Copy link

gwisp2 commented Jul 31, 2022

it doesn’t appear that command arguments via the compose file can be passed with this method

They can, I pass parameters in my docker-compose.

    command: ["-F", "mqtt://mqtt-broker,user=secret1,pass=secret2,events=rtl_433/events"]

Command is arguments to entrypoint script (or just a path to executable with arguments if entrypoint script is not defined).

@JB09
Copy link

JB09 commented Jul 31, 2022

They can, I pass parameters in my docker-compose.

Didn’t work for me. I even added an echo to your script and rebuilt the image and it’s not printing any variables for $@. Again it works with the .conf file but I’m not sure what the issue is.

@gwisp2
Copy link

gwisp2 commented Oct 3, 2022

Didn’t work for me. I even added an echo to your script and rebuilt the image and it’s not printing any variables for $@. Again it works with the .conf file but I’m not sure what the issue is.

Today I noticed that it doesn't work, lol. Probably I wanted to test that events were sent to mqtt but I was too lazy to do that and after a while I lost interest in tinkering home automation and forgot about that.

The problem is with ENTRYPOINT instruction. When a string X is passed to it then docker replaces it to ["sh", "-c", "X"].
As a result command-line options were passed to sh instead of the entrypoint script and for some reason sh ignores them instead of crashing with an error.

To avoid that use the following ENTRYPOINT. entrypoint.sh has +x permission and a shebang line so it can be executed directly.

ENTRYPOINT ["/opt/entrypoint.sh"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants