Camera Commands feedback

[jhansche]

https://github.com/mrlt8/docker-wyze-bridge/wiki/Camera-Commands

This is a work in progress and feedback would be appreciated

I wasn't sure how you wanted to receive this feedback, so I'm putting it here.

Send basic command to the camera over the WebUI's API or MQTT:

API: Make a GET request to /api//
MQTT: Publish a command to the topic wyzebridge//cmd
Can be disabled with DISABLE_CONTROL=True.

There are a few issues here that kind of get in the way of being able to use them effectively (at least in HA). Depending on use case, pretty much everything is possible... The question is how much effort will it take, and how many helpers and automations need to be cobbled together for it to work.

Generally speaking, there are 2-3 options for adding these commands to HA:

1. As a script: simple MQTT publish or resful.* service call. This is the least useful option, but also the least problematic. We can create a service for every individual command we want to trigger.
2. As an mqtt.switch (or .select): The available configuration options mostly work to allow exposing a switch here. But there are some downsides (see below)
3. As a restful.switch: Doesn't really work for setting the value, but could work as a restful.binary_sensor to show current state.

---

For mqtt.switch:

• There's no existing way to "ask" for the get_night_vision command in order to populate the initial state (or maybe I'm just not seeing it). So on HA startup, the state is "unknown" until you toggle it. The "state_topic" config will only listen to that topic, but doesn't have a way to ask for the initial value (i.e. by sending get_night_vision).
• The "command_topic" works fine for sending the payload_on/payload_off value for set_night_vision_off and set_night_vision_auto (or _on, if that's what you want).
• To make matters worse, since everything is all happening over one single topic (the commands we send, as well as the responses to those commands, for ALL possible commands), using state_topic to watch for the state_on or state_off values, is extremely fragile.
  For example, if you have one switch for night_vision and another switch for status_light, commands and responses for both of those will all be sent and received over a single topic, and that means the night vision mqtt.switch will also be monitoring the topic payloads (both outgoing and response) and receiving payloads for the status light switch. I don't see a way to do any topic payload filtering in the mqtt.switch integration, so all we can do is add conditions to the value_template, which WILL break whenever it sees a payload that belongs to a different command.

Here's an example of a generally working night vision switch:

mqtt:
  switch:
    - name: wyze cam night vision
      command_topic: wyzebridge/camera-name/cmd
      payload_off: "set_night_vision_off"
      payload_on: "set_night_vision_auto"
      
      # FIXME: same topic for send and receive
      state_topic: wyzebridge/camera_name/cmd

      # response: {"cmd": "get_night_vision", "status": "success", "response": "3"}
      # response: {"cmd": "set_night_vision_auto", "status": "success", "response": "1"}
      # What about other payloads (like the set* commands, or other non-night-vision commands)? How do we filter these?
      value_template: "{{ value_json.cmd == 'set_night_vision_auto' or (value_json.cmd == 'get_night_vision' and value_json.response in ['1', '3']) }}"
      # 1=on, 2=off, 3=auto
      state_on: True
      # FIXME: any other topic messages will collide and cause it to become "off" incorrectly.
      state_off: False
      icon: mdi:lightbulb-night-outline

This works as long as we only ever send and receive the exact commands listed here. Any other commands will break the reported state.

To solve this issue, would involve one or more of these changes:

1. Different incoming (command) topic vs outgoing (response) topics.
2. Different topics for command groups; e.g. PUBLISH topic=wyzebridge/<camera-name>/cmd/set_night_vision payload=auto; and LISTEN topic=wyzebridge/<camera-name>/cmd/get_night_vision (such that this topic will be published any time the night vision value changes 🤔

---

For a Restful entity, I didn't even get a chance to test it really, because the restful.switch integration doesn't seem to support a GET method for writing a state change - it only supports POST, PUT, and PATCH. It has options for body_on and body_off, but no option to change the resource (ie. URL) for the on or off states. So performing a GET http://bridge:5000/api/camera-name/set_night_vision_off vs set_night_vision_auto is not something that the restful component seems to be able to accomplish.

If we changed the API to allow POST/PUT/PATCH with a JSON command or value body, that would be way easier to support with the restful component. For example:

curl -X POST -v 'http://bridge:5000/api/camera-name/set_night_vision' -d '{"night_vision": "auto"}' # or '1|2|3'
curl -X PUT -v 'http://bridge:5000/api/camera-name/night_vision' -d 'off'  # or -d '1|2|3'
curl -X GET -v 'http://bridge:5000/api/camera-name/night_vision'  # response 1/2/3

This makes the endpoint more restful-friendly, and allows the restful.switch to work properly with it.

---

I know both of these significantly increase the complexity of both the REST API and the MQTT pub/sub architecture. But I just don't think that the current implementations work well with Home Assistant, or possibly other integrations for that matter.

To make this work properly, we would need to use something like an input boolean, paired with multiple automations (one to set the values over MQTT on manually changing, and one for triggering a state change from MQTT responses), since we are able to filter what MQTT payloads will allow an automation trigger to proceed, which we cannot do with mqtt.switch/etc.

We could probably do something similar as well with the REST API, using a combination of an input boolean, and multiple rest_command services for setting on/off/auto, and a restful.binary_sensor for obtaining the current state; plus automations to handle the input_boolean syncing.

So basically, it is possible to do ... it just takes a lot of "doing", to get it right.

Another way to achieve this would be a custom component that can look up the available wyze bridge cameras, and expose switches or buttons (etc) for all the available commands, backed by the mqtt or rest api.

---

I also don't see commands for toggling these options on the Pan Cam v3:

• Privacy Mode (rotates the camera straight down and turns off the camera feed)
• Pan Scan (cycles through waypoint presets, 10 seconds at a time)
• Track Motion (pans the camera to automatically follow motion)

[mrlt8]

Thanks for the feedback!

Would something like this work better?

REST

route('/api/<string:cam_name>/<string:cam_cmd>', methods=['GET', 'POST', 'PUT'])

POST /api/camera-name/night_vision -d '{"value": 3}'
PUT /api/camera-name/night_vision -d '3'
GET /api/camera-name/night_vision  # response 3

route('/api/<string:cam_name>/<string:cam_cmd>/<int:value>', methods=['GET'])

GET /api/camera-name/night_vision/3

Using an int value would allow us to send the value directly to the tutk protocol, however, it could get a little tricky for commands that take multiple inputs like rotation:

docker-wyze-bridge/app/wyzecam/tutk/tutk_protocol.py

Lines 691 to 709 in 920bc13

	class K11000SetRotaryByDegree(TutkWyzeProtocolMessage):
	"""
	Rotate by horizontal and vertical degree?
	Speed seems to be a constant 5.
	Parameters:
	- horizontal (int): horizontal position in degrees?
	- vertical (int): vertical position in degrees?
	- speed (int, optional): rotation speed. seems to default to 5.
	"""
	def __init__(self, horizontal: int, vertical: int, speed: int = 5):
	super().__init__(11000)
	self.horizontal = horizontal
	self.vertical = vertical
	self.speed = speed if 1 < speed < 9 else 5

It seems like we'll need to create a topic to set and get each command for MQTT:

command_topic: wyzebridge/camera-name/night_vision/set
payload_on: 3
payload_off: 2

state_topic: wyzebridge/camera_name/night_vision
state_on: 3
state_off: 2

As for the other commands, we would need to reverse those tutk commands, but I'm pretty sure privacy mode is set using the "power off" command which has to be sent over IoT Core's MQTT which is only accessible via the web API.

[jhansche]

Yes I think the night_vision example is exactly what I would expect, and nice that it supports both PUT and POST.

Would something like rotation work if the input is a JSON object, which gets converted to a python dict, and then passed as kwargs? So in your example, horizontal, vertical, and speed would be JSON/dict keys:

PUT /api/camera-name/rotation -d '{"horizontal": 45, "vertical": 30, "speed": 3}'

Then it can be invoked with:

# PUT -d'{"horizontal": 45, "vertical": 30, "speed": 3}'
# putArgsAsDict = {"horizontal": 45, "vertical": 30, "speed": 3}
msg = K11000SetRotaryByDegree(**putArgsAsDict)

With the current named kwargs, horizontal and vertical will be required in the dict, but speed would be optional because it has a default. If there's a way to get the current values, the PUT handler could fill those in if they're missing, or your PUT handler could respond with an error if it's missing. Or raise an error on PUT(/POST) since those should be complete resources according to REST, but allow a PATCH handler to fill in the missing h/v fields if necessary, since PATCH can allow partial resources?

It exposes some internal implementation details (the class's named kwargs have to be sent in the REST PUT request, so we need to know the required and optional argument names), but you could work around that by having the PUT handler map the keys.

The same approach could be used for the MQTT/set topics as well, at least for the complex setters like rotation: instead of payload being a single int value, it can be a JSON object of named kwargs.

As for MQTT, yes, I think that having separate topics for each get and set command, would be ideal, and is pretty typical for IOT MQTT usages. The MQTT docs have several examples of what typical device+command topics look like. If each command has its own get/set topic, it makes it extremely easy to create those devices automatically from the add-on, by sending discovery messages for each one, and the entities will simply appear in HA.

In fact, after I sent this feedback, I looked through my entities, and I do see several wyze_*_audio, wyze-*_night_vision, etc, entities, which seem to have been created by the docker wyze bridge, but they're all marked unavailable now (maybe it was being tested before and I didn't realize it - and looking at mqtt.py now, I do see discovery here, which would explain why I have the entities, but not sure why they don't work). With this new topic structure, that discovery would work more effectively, by giving each command and state its own topic, to avoid collisions.

And on your end, you can still subscribe to a wildcard topic, and just parse the destination/command out of the received message's topic; or subscribe to each incoming topic individually, to give each one its own handler.

[jhansche]

As for the other commands, we would need to reverse those tutk commands, but I'm pretty sure privacy mode is set using the "power off" command which has to be sent over IoT Core's MQTT which is only accessible via the web API.

I've verified it's done using action_key=power_on and power_off. But it looks like it goes to api.wyzecam.com/app/v2/auto/run_action - is that the same as "IoT Core's MQTT"? Also not sure what you mean by "via the web API" - is that referring to the port 5000 Web UI, or the "wyze web api"? (EDIT: or is this the difference between LAN control vs API control, and we'd want to prefer LAN control if possible, which means IOTC MQTT?)

I noticed that when it's powered off, the Web UI doesn't seem to be able to turn it back on. I tried "Start" and it just responds with an error, and this appears in the logs:

[pan-2] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Pan 2 is offline.

I understand it's offline because I turned it off (privacy mode). But does that mean there's no way in the bridge to turn it on? If we're able to hit this run_action API, is it just a matter of adding the command for it to work, or is there some other reason preventing it from working? Would the request/response info from that endpoint be helpful for this?

I also have details for the other commands mentioned: get/set Motion Tracking, and toggle Pan Scan (cruise). Is it just a matter of creating the tutk_protocol implementations for it to be supported? I don't mind adding those if you're interested.

[mrlt8]

From what I can tell, the cameras are running an MQTTS client that connects to AWS IoT Core. The app can communicate with the camera locally using TUTK but has to use the wyze web api (like api.wyzecam.com/app/v2/auto/run_action) for the IoT Core stuff.

Unfortunately, some commands like power seem to go over the wyze api and sending a power off command pretty much just disables the tutk portion, so we don't have a way to power it back on unless we use the wyze/IoT Core API. However, wyze doesn't seem to like people accessing their API and has threatened to block some of the other HA projects, so I try to avoid pinging it as much as possible. (They've actually started requiring a signature for their v3 api. #793)

MQTT discovery seems great, but I could never get HA to accept the Camera Entity with the stream options. Open to suggestions or ideas!

As for updating some of the entities, we could always use K10020 to lookup multiple values from the camera. I've added a camera_info MQTT/REST GET topic that should return some values as well as a param_info topic that can accept a comma separated string as the payload to lookup specific values. e.g.:

topic: wyzebridge/storage/param_info/get
payload: 50

will return the irled value {"50": 2}

topic: wyzebridge/storage/param_info/get
payload: 3,5

would return the bitrate and fps {"3": 180, "5": 20}

[jhansche]

Ok, so if I understand correctly:

• TUTK is the local LAN app<->cam channel?
• And IOTC is the "cloud" channel, which has separate transports: MQTT for cam<->iotc, and HTTP (a la run_action) for app<->iotc?

Assuming those are correct:

• IOTC can be used to send power_off, but cannot be used to send power_on? My network trace shows that that's what the app does when coming out of privacy mode. Or you're saying we can use it to send both off and on, but it requires going over the IOTC API and you want to minimize that to minimize exposure and risk of wyze shutting it down?
• And there is no way to send power_off/power_on over the local TUTK channel, only IOTC? Or at least no way that we know of

I think I'm starting to get a better understanding of how these all work together. Would be cool if there was a wiki page that describes the architecture in more detail, or at least a glossary of terms and what they mean, or even a diagram that shows the distinctions. Also a bit confusing because the P2P Mode section describes "P2P" as going over the internet, and that is not my understanding of "P2P" which I understand to mean "peer-to-peer", which means there is no need for a 3rd party MITM, but the description of P2P mode implies that it uses Wyze/AWS as the MITM.

I think I'm getting off-topic though. Maybe I can make a separate PR to the wiki to put down my understanding of how it all works, and let you review that 😄

---

As for K10020, if I'm understanding correctly, you're pointing this out as a way to remove the need to use the wyzebridge/camera_name/cmd/night_vision/get (et al) topics, if we can use the get_param topic instead? I think that makes sense if your goal is to display a summary of the current state of the camera. However I don't think that helps in terms of a single MQTT Switch for "Night Vision" or "Status LED" or "IR LED", etc. Especially with MQTT discovery where the wyzebridge sends the discovery config for each individual entity, it's actually best to keep each entity fully self-contained: where it cannot accidentally modify another entity's state, and will not accidentally receive state of another entity.

But maybe I'm misunderstanding the goal of the K10020 topic?

And as for MQTT Discovery of the Camera Entity, I'd have to see what you started with, and what issues you encountered. Do you have a WIP commit or github issue somewhere that shows the issues? I can try to test that to see if I can come up with a workaround.

FWIW, I wouldn't even be using the MQTT Camera myself, because I use Frigate to consume the camera streams. The other entities would just be for controlling the camera's features, separately from the camera streams. For example, I have a working MQTT switch right now for toggling Night Vision, and I use that in automations to turn Night Vision off during the day, and turn it on when we turn off the lights. I do this because Wyze's auto night vision mode is not reliable, and often switches to night vision mode too early. The camera entity that we use to show the stream actually comes from Frigate with object detection and so on; but Frigate can't control the night vision mode (or pan rotation, etc).

[mrlt8]

The IOTC API is part of TUTK. Older TUTK documentation is available online with lots of info: https://www.sunipcam.com/sdk/Readme.htm

From what I understand, the app mostly uses TUTK for video and a few local commands and the web api with run_action as a sort of interface for AWS IOT for the notifications and all the other commands.

The power on/off commands are sent over the web API and seem to just disable TUTK on the camera which in theory "disables" the video, though, some of the cameras actually have another KVS server for the Alexa/google integrations.

Most of the other "wyze api" projects have reversed the web api and use that to communicate with the wyze devices:
https://github.com/shauntarves/wyze-sdk
https://github.com/JoshuaMulliken/wyzeapy

However, I believe some of these projects were polling the web api endpoints which is part of the reason why wyze now has a bunch of rate limits.

---

As for K10020, I've added an option to update the MQTT topics on initial connection

docker-wyze-bridge/app/wyzebridge/wyze_control.py

Lines 181 to 189 in 47f6a7a

	param = send_tutk_msg(sess, ("param_info", "2,50"), False).get("param_info")
	if param and (resp := param.get("response")):
	send_mqtt(
	[
	(f"wyzebridge/{sess.camera.name_uri}/night_vision", resp.get("2", 0)),
	(f"wyzebridge/{sess.camera.name_uri}/irled", resp.get("50", 0)),
	]
	)

I also tweaked the MQTT discovery so that the controls will remain active even if the camera disconnects since it will now connect to the camera to send the message.