Report Position of Curtains After Sending Stop Command #31

fphammerle · 2021-05-13T06:16:18Z

fphammerle · 2021-05-13T07:18:22Z

I do not own a switchbot curtain motor that I could use for testing.

@oetelaar24, could you maybe install this pre-release https://pypi.org/project/switchbot-mqtt/0.7.0a0/#files
and check if you receive the motor's position on topic homeassistant/cover/switchbot-curtain/MAC_ADDRESS/position
after sending a stop command?

If that works as expected, I suggest we discuss when the position should be reported (constant interval? upon request?) here: #30

oetelaar24 · 2021-05-13T17:16:44Z

Hi @fphammerle,

Unfortunatly, it reports an error:

2021-05-13T17:08:11+0000:DEBUG:switchbot:Connecting to Switchbot...
2021-05-13T17:08:11+0000:DEBUG:switchbot:Connected to Switchbot.
2021-05-13T17:08:11+0000:DEBUG:switchbot:Prepare to send
2021-05-13T17:08:12+0000:DEBUG:switchbot:Sending command, 570F450100ff
2021-05-13T17:08:12+0000:INFO:switchbot:Successfully sent command to Switchbot (MAC: CE:CF:88:xx:xx:xx).
2021-05-13T17:08:12+0000:DEBUG:switchbot:Disconnecting
2021-05-13T17:08:12+0000:INFO:switchbot_mqtt:switchbot curtain CE:CF:88:xx:xx:xx stopped
2021-05-13T17:08:12+0000:DEBUG:switchbot_mqtt:publishing topic=homeassistant/cover/switchbot-curtain/CE:CF:88:xx:xx:xx/state payload=b''
Traceback (most recent call last):
  File "/switchbot-mqtt/.venv/bin/switchbot-mqtt", line 8, in <module>
    sys.exit(_main())
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot_mqtt/__init__.py", line 336, in _main
      
(
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot_mqtt/__init__.py", line 301, in _run
    mqtt_client.loop_forever()
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 1779, in loop_forever
    rc = self.loop(timeout, max_packets)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 1572, in loop_read
    rc = self._packet_read()
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 2310, in _packet_read
    rc = self._packet_handle()
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 2936, in _packet_handle
    return self._handle_publish()
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 3216, in _handle_publish
    self._handle_on_message(message)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/paho/mqtt/client.py", line 3429, in _handle_on_message
    callback(self, self._userdata, message)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot_mqtt/__init__.py", line 92, in _mqtt_command_callback
    cls(mac_address=mac_address).execute_command(
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot_mqtt/__init__.py", line 262, in execute_command
    self._update_position(mqtt_client=mqtt_client)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot_mqtt/__init__.py", line 231, in _update_position
    self._device.update()
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/switchbot/__init__.py", line 189, in update
    devices = bluepy.btle.Scanner().scan(scan_timeout)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/bluepy/btle.py", line 852, in scan
    self.start(passive=passive)
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/bluepy/btle.py", line 790, in start
    self._mgmtCmd("le on")
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/bluepy/btle.py", line 312, in _mgmtCmd
    raise BTLEManagementError("Failed to execute management command '%s'" % (cmd), rsp)
bluepy.btle.BTLEManagementError: Failed to execute management command 'le on' (code: 20, error: Permission Denied)
2021-05-13T17:08:24+0000:INFO:switchbot_mqtt:connecting to MQTT broker xx.xx:1883
2021-05-13T17:08:24+0000:DEBUG:switchbot_mqtt:connected to MQTT broker x.x.x.x:1883
2021-05-13T17:08:24+0000:INFO:switchbot_mqtt:subscribing to MQTT topic 'homeassistant/switch/switchbot/+/set'
2021-05-13T17:08:24+0000:INFO:switchbot_mqtt:subscribing to MQTT topic 'homeassistant/cover/switchbot-curtain/+/set'

I had to run the pip install command inside my current docker container (0.6.0-armv6) to get the new release.
I'm not sure if that is the problem here. If needed I could try it straight onto the RBPi3B+ itself.

fphammerle · 2021-05-13T21:27:07Z

Thanks a lot for testing and posting the log, @oetelaar24 !

Apparently scanning (bluepy.btle.Scanner().scan(scan_timeout) in https://github.com/Danielhiversen/pySwitchbot/blob/0.10.0/switchbot/__init__.py#L189) requires "more" privileges than sending commands (https://github.com/Danielhiversen/pySwitchbot/blob/0.10.0/switchbot/__init__.py#L91)

The easiest fix would probably be running the container as root / uid=0 (currently nobody: https://github.com/fphammerle/switchbot-mqtt/blob/v0.7.0a0/Dockerfile#L62), but I would like to avoid that for security reasons

I assume that you would get the same error when running "outside" the container with uid != 0.

I'll try to find a different solution

oetelaar24 · 2021-05-14T07:07:29Z

Thanks for this extra information!
If you want, I can try to test it further?

Reading in on the issue, I saw this comment which lets me believe there is a way around running the container as root: IanHarvey/bluepy#218 (comment)

fphammerle · 2021-05-15T19:29:37Z

Thanks for your offer to test and suggestion!

Unfortunately, sudo setcap 'cap_net_raw,cap_net_admin+eip' bluepy-helper is incompatible with docker run --security-opt no-new-privileges.

Enabling the no-new-privileges options is generally recommended to keep unprivileged processes from becoming root / gaining capabilties:
https://github.com/docker/docker-bench-security/blob/v1.3.5/tests/2_docker_daemon_configuration.sh#L410
https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html#rule-4-add-no-new-privileges-flag
https://jfrog.com/knowledge-base/docker-image-security-6-must-know-tips/

fphammerle · 2021-05-22T09:46:41Z

For reference:

The permission errors gets raised in bluepy.btle.Scanner().start()
( https://github.com/IanHarvey/bluepy/blob/v/1.3.0/bluepy/btle.py#L787 )
and can be triggered via:

>>> import bluepy
>>> scanner = bluepy.btle.Scanner()
>>> scanner._startHelper(iface=scanner.iface)
>>> scanner._mgmtCmd("le on")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/switchbot-mqtt/.venv/lib/python3.8/site-packages/bluepy/btle.py", line 312, in _mgmtCmd
    raise BTLEManagementError("Failed to execute management command '%s'" % (cmd), rsp)
bluepy.btle.BTLEManagementError: Failed to execute management command 'le on' (code: 20, error: Permission Denied)

or:

$ sudo docker run --rm -it --userns host --network host \
    fphammerle/switchbot-mqtt@sha256:26255bd711621037b2886407f29ae5ce3453b44f1322e717f4d1e31f30f85b85 \
    /switchbot-mqtt/.venv/lib/python3.8/site-packages/bluepy/bluepy-helper
# bluepy-helper.c version 1.3.0 built at 08:12:57 on May 22 2021
le on
rsp=$mgmtcode=$mgmterrestat=h14emsg='Permission Denied

syscalls of bluepy-helper:

[...]
read(0, "le on\n", 1024)    = 6
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
poll([{fd=0, events=POLLIN}, {fd=3, events=POLLIN|POLLOUT}, {fd=4, events=POLLIN}], 3, -1) = 2 ([{fd=3, revents=POLLOUT}, {fd=4, revents=POLLIN}])
read(4, "\3\0\0\0\0\0\0\0", 16) = 8
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
writev(3, [{iov_base="\r\0\0\0\1\0\1", iov_len=7}], 1) = 0
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
poll([{fd=0, events=POLLIN}, {fd=3, events=POLLIN}, {fd=4, events=POLLIN}], 3, -1) = 2 ([{fd=3, revents=POLLIN}, {fd=4, revents=POLLIN}])
read(4, "\3\0\0\0\0\0\0\0", 16) = 8
write(4, "\1\0\0\0\0\0\0\0", 8) = 8
read(3, "\2\0\0\0\3\0\r\0\24", 512) = 9
writev(1, [{iov_base="rsp=$mgmt\36code=$mgmterr\36estat=h14\36emsg='Permission Denied", iov_len=57}, {iov_base="\n", iov_len=1}], 2) = 58
[...]

fd/3 is the bluetooth socket:

socket(AF_BLUETOOTH, SOCK_RAW|SOCK_CLOEXEC|SOCK_NONBLOCK, BTPROTO_HCI) = 3
bind(3, {sa_family=AF_BLUETOOTH, hci_dev=htobs(65535), hci_channel=HCI_CHANNEL_CONTROL}, 6) = 0

fd/4:

eventfd2(0, EFD_CLOEXEC|EFD_NONBLOCK) = 4

fphammerle · 2021-05-22T13:38:43Z

related question: is it possible to run hcitool lescan with uid!=0 and CapEff==0?

$ sudo strace -e trace=writev hcitool lescan
writev(3, [{iov_base="\1", iov_len=1}, {iov_base="\v \7", iov_len=3}, {iov_base="\1\20\0\20\0\0\0", iov_len=7}], 3) = 11
writev(3, [{iov_base="\1", iov_len=1}, {iov_base="\f \2", iov_len=3}, {iov_base="\1\1", iov_len=2}], 3) = 6
LE Scan ...
[...]
$ strace -e trace=writev hcitool lescan
writev(3, [{iov_base="\1", iov_len=1}, {iov_base="\v \7", iov_len=3}, {iov_base="\1\20\0\20\0\0\0", iov_len=7}], 3) = -1 EPERM (Operation not permitted)
Set scan parameters failed: Operation not permitted
+++ exited with 1 +++

apparently, CAP_NET_RAW is required:

$ sudo setcap cap_net_raw+ep /usr/bin/hcitool
$ strace -e trace=writev hcitool lescan
writev(3, [{iov_base="\1", iov_len=1}, {iov_base="\v \7", iov_len=3}, {iov_base="\1\20\0\20\0\0\0", iov_len=7}], 3) = 11
writev(3, [{iov_base="\1", iov_len=1}, {iov_base="\f \2", iov_len=3}, {iov_base="\1\1", iov_len=2}], 3) = 6
LE Scan ...
[...]

stack trace:

(gdb) catch syscall writev
Catchpoint 1 (syscall 'writev' [146])
(gdb) r
Starting program: /usr/bin/hcitool lescan
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Catchpoint 1 (call to syscall writev), 0xb6ed1618 in __GI___writev (fd=fd@entry=3, iov=iov@entry=0xbefff7b4, 
    iovcnt=iovcnt@entry=3) at ../sysdeps/unix/sysv/linux/writev.c:26
26	../sysdeps/unix/sysv/linux/writev.c: No such file or directory.
(gdb) bt
#0  0xb6ed1618 in __GI___writev (fd=fd@entry=3, iov=iov@entry=0xbefff7b4, iovcnt=iovcnt@entry=3)
    at ../sysdeps/unix/sysv/linux/writev.c:26
#1  0x0001ead4 in hci_send_cmd (dd=dd@entry=3, ogf=<optimized out>, ocf=<optimized out>, plen=<optimized out>, param=0xbefff964)
    at lib/hci.c:1113
#2  0x0001ec0c in hci_send_req (dd=dd@entry=3, r=0xbefff96c, r@entry=0xbefff964, to=10000, to@entry=0) at lib/hci.c:1143
#3  0x0002210c in hci_le_set_scan_parameters (dd=dd@entry=3, type=type@entry=1 '\001', interval=interval@entry=0, 
    window=window@entry=0, own_type=0 '\000', own_type@entry=228 '\344', filter=filter@entry=0 '\000', to=to@entry=10000)
    at lib/hci.c:2967
#4  0x00016934 in cmd_lescan (dev_id=0, dev_id@entry=-1, argc=<optimized out>, argc@entry=1, argv=<optimized out>, 
    argv@entry=0xb6ffd100) at tools/hcitool.c:2558
#5  0x00011200 in main (argc=1, argv=<optimized out>) at tools/hcitool.c:3491

https://github.com/bluez/bluez/blob/5.50/tools/hcitool.c#L2558

oetelaar24 · 2021-06-22T11:23:08Z

Thanks for looking into it and digging around.
Is there anything I can do at this point to help you?

fphammerle · 2021-06-22T14:54:09Z

Thanks! Unfortunately, I haven't found a workaround for the issue described above yet.
In summary the open question is: Does linux provide a way to scan for bluetooth low energy devices that is compatible with the no-new-privileges flag?

oetelaar24 · 2021-09-09T21:22:18Z

Hi, I see that in #37, you mention

In the meantime, adding --security-opt seccomp=unconfined to docker run might fix the issue, but on the downside that option is disabling some of docker's security feaetures.

Would that also work for these issues? I mean, I only run it on a RPi anyway so i'm not too worried about the security implications...

rebased ab364bc

rebased 885d557

…or's position (can't enable by default due to `bluepy.btle.BTLEManagementError: Failed to execute management command 'le on'`)

…` triggers `BTLEManagementError` #31 (comment)

… setups #31 (comment)

fphammerle · 2021-09-11T08:11:57Z

@oetelaar24, could you upgrade to the new pre-release v1.1.0a0 and add the command-line flag --fetch-device-info?

You should then see the following instructions how to workaround the issue:

PermissionError: bluepy-helper failed to enable low energy mode due to insufficient permissions.
[...]
Insecure workaround:
1. sudo apt-get install --no-install-recommends libcap2-bin
2. sudo setcap cap_net_admin+ep /somewhere/bluepy/bluepy-helper
3. restart switchbot-mqtt
In docker-based setups, you could use `sudo docker run --cap-drop ALL --cap-add NET_ADMIN --user 0 …` (seriously insecure).

After applying the workaround you should get position reports on the topic homeassistant/cover/switchbot-curtain/MAC_ADDRESS/position after sending a stop command.

…ON_TOPIC_LEVELS

oetelaar24 · 2021-09-29T16:40:53Z

@fphammerle could you publish an armv6 image of this new pre-release please? My RPi3B only runs armv6 unfortunately...

fphammerle · 2021-09-29T20:10:10Z

Sorry @oetelaar24, I forgot about that. Pushed now: docker.io/fphammerle/switchbot-mqtt:1.1.0a0-armv6 (digest: sha256:8701cbd571c0cc4f89aaa2dec478fa5638eb0ce4f3c479f4ab99885153075727)

oetelaar24 · 2021-10-05T14:36:43Z

It seems to work now:

2021-10-05T14:29:46+0000:DEBUG:switchbot:Sending command to switchbot 57xxxxx00ff
2021-10-05T14:29:46+0000:DEBUG:switchbot:Connecting to Switchbot...
2021-10-05T14:29:46+0000:DEBUG:switchbot:Connected to Switchbot.
2021-10-05T14:29:46+0000:DEBUG:switchbot:Prepare to send
2021-10-05T14:29:47+0000:DEBUG:switchbot:Sending command, 57xxxxx00ff
2021-10-05T14:29:47+0000:INFO:switchbot:Successfully sent command to Switchbot (MAC: CE:CF:88:3F:XX:XX).
2021-10-05T14:29:47+0000:DEBUG:switchbot:Disconnecting
2021-10-05T14:29:47+0000:INFO:switchbot_mqtt:switchbot curtain CE:CF:88:3F:XX:XX stopped
2021-10-05T14:29:47+0000:DEBUG:switchbot_mqtt:publishing topic=homeassistant/cover/switchbot-curtain/CE:CF:88:3F:XX:XX/state payload=b''
2021-10-05T14:29:52+0000:DEBUG:switchbot_mqtt:publishing topic=homeassistant/cover/switchbot-curtain/CE:CF:88:3F:XX:XX/position payload=b'47'

the position value is somewhat different from the one in the app tho.

EDIT: A I get it now, it's reverse of what it says in the app, so 53% opened, is 47% closed. Logical :)

I tried adding a position_topic to the cover configuration in hassio, however it doesn't seem to work just yet, I might be doing something wrong :-P

- platform: mqtt
  name: Schuifpui Gordijn
  unique_id: Schuifpui_Gordijn
  command_topic: homeassistant/cover/switchbot-curtain/CE:CF:88:3F:XX:XX/set
  state_topic: homeassistant/cover/switchbot-curtain/CE:CF:88:3F:XX:XX/state
  position_topic: homeassistant/cover/switchbot-curtain/CE:CF:88:3F:XX:XX/position
  device_class: curtain

fphammerle · 2021-10-06T06:12:38Z

Thanks a lot @oetelaar24 for all your tests!

I'll release this as v1.1.0.

oetelaar24 · 2021-10-06T07:49:25Z

Happy to help! @fphammerle !
Is there any further things I can do?
I see that the position gets reported nicely on stop. Can you however also report position for instance every 5 secs after an open/close command for one minute or so? That way we would see some progress and also get a nice open/close status when the curtains are done moving without a stop command being sent :) #thinkingOutLoud :)

fphammerle · 2021-10-23T16:30:16Z

Please see #54 (comment)

fphammerle force-pushed the report-curtain-state branch 3 times, most recently from 05b8b46 to ab364bc Compare May 13, 2021 06:34

fphammerle changed the title ~~Report Position of Curtains~~ Report Position of Curtains After Sending Stop Command May 13, 2021

fphammerle force-pushed the report-curtain-state branch from 424764e to 885d557 Compare May 13, 2021 07:05

fphammerle mentioned this pull request Jun 28, 2021

Switchbot Password & Switchbot Homeassistant State #37

Closed

fphammerle mentioned this pull request Jul 26, 2021

Switchbot battery #41

Closed

fphammerle added 2 commits September 10, 2021 20:48

implement _CurtainMotor._report_position()

794d14c

rebased ab364bc

curtains: report position after sending stop command

0e9729f

rebased 885d557

fphammerle force-pushed the report-curtain-state branch from 885d557 to 0e9729f Compare September 10, 2021 18:53

fphammerle added 2 commits September 10, 2021 23:10

added option --fetch-device-info to toggle reporting of curtain mot…

90d274d

…or's position (can't enable by default due to `bluepy.btle.BTLEManagementError: Failed to execute management command 'le on'`)

provide further information & workaround in exception msg when `le on…

fadf387

…` triggers `BTLEManagementError` #31 (comment)

fphammerle force-pushed the report-curtain-state branch 2 times, most recently from c96e826 to 3a614dd Compare September 11, 2021 06:33

fix capturing of BTLEManagementError on pySwitchbot v0.10.1

22d3f7b

fphammerle force-pushed the report-curtain-state branch from 3a614dd to 22d3f7b Compare September 11, 2021 06:37

BTLEManagementError: added workaround instructions for docker-based…

81ab456

… setups #31 (comment)

refactor to remove duplicate definition of _CurtainMotor._MQTT_POSITI…

c7e77dc

…ON_TOPIC_LEVELS

fphammerle marked this pull request as ready for review October 6, 2021 05:50

fphammerle merged commit c7e77dc into master Oct 6, 2021

fphammerle deleted the report-curtain-state branch October 6, 2021 06:11

fphammerle mentioned this pull request Oct 23, 2021

device info requests #54

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Report Position of Curtains After Sending Stop Command #31

Report Position of Curtains After Sending Stop Command #31

fphammerle commented May 13, 2021

fphammerle commented May 13, 2021

oetelaar24 commented May 13, 2021 •

edited

Loading

fphammerle commented May 13, 2021

oetelaar24 commented May 14, 2021

fphammerle commented May 15, 2021

fphammerle commented May 22, 2021

fphammerle commented May 22, 2021 •

edited

Loading

oetelaar24 commented Jun 22, 2021

fphammerle commented Jun 22, 2021

oetelaar24 commented Sep 9, 2021

fphammerle commented Sep 11, 2021 •

edited

Loading

oetelaar24 commented Sep 29, 2021

fphammerle commented Sep 29, 2021

oetelaar24 commented Oct 5, 2021 •

edited

Loading

fphammerle commented Oct 6, 2021

oetelaar24 commented Oct 6, 2021

fphammerle commented Oct 23, 2021

Report Position of Curtains After Sending Stop Command #31

Report Position of Curtains After Sending Stop Command #31

Conversation

fphammerle commented May 13, 2021

fphammerle commented May 13, 2021

oetelaar24 commented May 13, 2021 • edited Loading

fphammerle commented May 13, 2021

oetelaar24 commented May 14, 2021

fphammerle commented May 15, 2021

fphammerle commented May 22, 2021

fphammerle commented May 22, 2021 • edited Loading

oetelaar24 commented Jun 22, 2021

fphammerle commented Jun 22, 2021

oetelaar24 commented Sep 9, 2021

fphammerle commented Sep 11, 2021 • edited Loading

oetelaar24 commented Sep 29, 2021

fphammerle commented Sep 29, 2021

oetelaar24 commented Oct 5, 2021 • edited Loading

fphammerle commented Oct 6, 2021

oetelaar24 commented Oct 6, 2021

fphammerle commented Oct 23, 2021

oetelaar24 commented May 13, 2021 •

edited

Loading

fphammerle commented May 22, 2021 •

edited

Loading

fphammerle commented Sep 11, 2021 •

edited

Loading

oetelaar24 commented Oct 5, 2021 •

edited

Loading