Add support for floodlight control over MQTT #14

kevin-david · 2022-12-11T14:01:46Z

This works... sometimes, but needs some more work/testing.

crates/core/src/bc_protocol/floodlight_status.rs

QuantumEntangledAndy · 2022-12-12T05:17:24Z

So what is up with the sometimes? Are there any telling errors?

QuantumEntangledAndy · 2022-12-12T05:18:31Z

Maybe start with a simpl cmd command rather than the mqtt interface so we can check if it the actual status light bit is working without the extra layer of difficulty that mqtt brings

QuantumEntangledAndy · 2022-12-13T13:57:49Z

How's it going I finished my rework of the pausable streams and I made some changes to mqtt not sure how that effects you so thought I'd check in here. My changes are currently on #17

kevin-david · 2022-12-13T14:09:26Z

Unfortunately I haven't been able to make time to look into this yet - I will try your suggestion next of just starting with a simple command, I think that's a good idea.

QuantumEntangledAndy · 2022-12-13T14:19:19Z

Alright I'll tackle mqtt next then and make sure everything is working there. Good luck with your hacking and let me know if you need some help. I recommend checking out src/pir/*, src/main.rs and src/cmdline.rs as those will take you though how to add a simple command

QuantumEntangledAndy · 2022-12-14T04:08:53Z

Sorry for the odd question but is your floodlight battery powered? I ask because I want to work out the camera messages for battery levels but I only own a non-battery camera.

kevin-david · 2022-12-14T13:43:51Z

Sorry for the odd question but is your floodlight battery powered? I ask because I want to work out the camera messages for battery levels but I only own a non-battery camera.

It is not - PoE powered! Here's the product page: https://reolink.com/product/reolink-floodlight/

QuantumEntangledAndy · 2022-12-14T14:01:11Z

Ah I see thanks anyway.

QuantumEntangledAndy · 2023-01-05T04:17:35Z

Did you ever find time to work on this? If not I can work something up.

kevin-david · 2023-01-06T21:49:39Z

Unfortunately I haven't - I was kind of waiting for an epiphany or Reolink to bail us out like I mentioned in the comment here: fwestenberg/reolink_dev#599 (comment)

I do want (more) control of the floodlight/spotlight on the Duo Floodlight PoE camera I just got though - I am hoping the protocol is the same even if it's not exposed, specifically for the time the light is kept on after motion is detected, since according to Reolink that's "related to the duration of the alarm and the post-motion recording you set" right now and not able to be set on its own.

I am not seeing a lot of free time opening up in my schedule soon though :(

kevin-david · 2023-01-25T04:43:35Z

@QuantumEntangledAndy OK i am back at it here because I want to make my floodlight turn on another one :)

A few things that need to be changed...

Message ID 291 (MSG_ID_FLOODLIGHT_STATUS_LIST) can't be written, only read. A lot like 33 (MSG_ID_MOTION) - so all the stuff I added in get_floodlight_status is wrong, the camera returns a 405 (Method Not Allowed).
Need to deal with [2023-01-25T04:29:14Z DEBUG neolink_core::bc_protocol::connection::bcconn] Ignoring uninteresting message num 291 - which is actually interesting
- I think one way to approach this would be to copy how listen_on_motion works, where (based on my read) we set up a thread to process incoming messages...
  - However, I am not really sure we need this - can we just add a subscriber here somehow? I looked through the code and it wasn't obvious:
    
    neolink/crates/core/src/bc_protocol/connection/bcconn.rs
    
    Line 134 in 2e6f9c7
    
    subscribers: &mut Arc<Mutex<BTreeMap<u16, Sender<Bc>>>>,

Finally, the floodlight seems to drop off after some time, even with no commands being sent. Each time, it looks like:

[2023-01-25T04:30:25Z ERROR neolink_core::bc_protocol::connection::bcconn] Deserialization error: Deserialization error
[2023-01-25T04:30:27Z ERROR neolink_core::bc_protocol::connection::bcconn] caused by: I/O error
[2023-01-25T04:30:27Z ERROR neolink_core::bc_protocol::connection::bcconn] caused by: Read returned 0 bytes
[2023-01-25T04:30:27Z ERROR neolink::mqtt::event_cam] Motion thread aborted: Dropped connection
    
    Caused by:
        receiving on an empty and disconnected channel

What do you think about adding retry logic here, or making this a non-fatal error somehow? AFAIK from other languages, reading 0 bytes means "we're done" so that's a bit odd, maybe a quirk of this device?

QuantumEntangledAndy · 2023-01-25T08:59:54Z

This would be easier if I had some wireshark to look at.

There needs to be a subscriber somewhere to know where to the send the reply.

When you send a Bc packet you create a new message number. The camera will reply with the same message number. If your missing the subscriber or the subscriber is dropped then you get the message you described. I'll try and check your code and understand why you are not keeping your subscriber around long enough.

What does the official message flow look like. Do you request the status and then wait for a single reply or multiple replies?

QuantumEntangledAndy · 2023-01-25T09:16:28Z

I think you need to have a look at the offical client again in wireshark and find a message with ID MSG_ID_FLOODLIGHT_STATUS_LIST then filter the messages by msg_num for that message. It should show you the whole communication thread for that message from beginning to end.

QuantumEntangledAndy · 2023-01-25T09:19:50Z

What do you think about adding retry logic here, or making this a non-fatal error somehow? AFAIK from other languages, reading 0 bytes means "we're done" so that's a bit odd, maybe a quirk of this device?

If the connection is dropped we would have to start the whole connection from scratch and relogin. It is possible that there is a keep-alive message we could send peridocally. We have such a thing for the UDP connections but perhaps this device needs it even over TCP. Again seeing the wireshark would help

kevin-david · 2023-01-26T01:49:03Z

Packet captures attached: pcaps-reolink-floodlight.zip

I'll try and check your code and understand why you are not keeping your subscriber around long enough.

I'm not sure I'm setting it up properly at all, to be clear. I will keep digging in.

What does the official message flow look like. Do you request the status and then wait for a single reply or multiple replies?

As far as I can tell, there's a single request for multiple replies. At the end of the floodlight_comms_reolink_app_2 capture, you can see where I turn on and off the light with my phone (another client)

I think you need to have a look at the offical client again in wireshark and find a message with ID MSG_ID_FLOODLIGHT_STATUS_LIST then filter the messages by msg_num for that message. It should show you the whole communication thread for that message from beginning to end.

As far as I can tell (desktop client) this message is never sent client->camera, only camera->client. I can try to get a mobile app pcap again if you think that'd be useful.

It is possible that there is a keep-alive message we could send peridocally. We have such a thing for the UDP connections but perhaps this device needs it even over TCP. Again seeing the wireshark would help

I think you're right. From my read of the captures, the <LinkType> message (93) is being used as a keepalive, the TCP keep alives being sent by neolink aren't enough AFAICT.

QuantumEntangledAndy · 2023-01-26T02:27:19Z

Your messages seem atypical. Usually the message_handle field in the header links the message and the reply however in your dumps they are all 0 instead you have the channel_id doing what seems to be the usual msg_id's job.

Usually it works like this:

Client sends: msg_handle 7, channel_id: 0, payload HDDInfoList
Camera replies: msg_handle 7, channel_id: 0, payload HDDInfoList

To match the reply with the message sent the msg_handle is used as a tag.

However your reolink app is doing it like this:

Client sends: msg_handle 0, channel_id: 7, payload HDDInfoList
Camera replies: msg_handle 0, channel_id: 7, payload HDDInfoList

Your type 291's are also all on channel_id: 0 which means it is in reply to login. You'd need to subscribe to message ID 0 to capture them assuming that the floodgate does in fact send them when reolink connects.

It may also be that we are getting a disconnect from the floodgate because the message_handle is non-zero in neolink and the camera is complaining

QuantumEntangledAndy · 2023-01-26T02:32:13Z

Can you send two additional wireshark dumps:

The camera using the Manual Floodlight Control message
A full dump of neolink talking to the floodlight

QuantumEntangledAndy · 2023-01-26T02:45:18Z

I'm going to see if I can wrap up my rework into async code and UDP improvements then come and tackle this one with you. In the mean time you can try subscribing to msg_handle 0 and see what you can get

kevin-david · 2023-01-26T03:16:39Z

The camera using the Manual Floodlight Control message

Here you go, this is from the Android app, not the Mac app like I shared before: reolink_app_wireshark.zip. Manual control is message type 288, with 291 coming as a response after sending. <LinkType>/93 seems to be used as keepalives here as well

I accidentally managed to capture 2 versions because the IP of the device changed. When it couldn't contact it at 192.168.1.130, the app managed to find it at 192.168.1.131 and started communicating over UDP. Once I removed and readded the camera in the app, TCP worked. I removed the login messages in an attempt to not have to rotate my passwords a bunch of times, but if you need that let me know.

A full dump of neolink talking to the floodlight

This was in the original ZIP - it's floodlight_comms_neolink_mqtt_2.pcapng. Happy to get another one with a different command if you want.

kevin-david · 2023-01-26T03:54:45Z

I was able to get this partially working again! Publishing an MQTT message to the broker (i.e. neolink/backyard floodlight/control/floodlight set to raw on or off works until the camera disconnects due to the missing <LinkType> keepalives.

So the big things missing are:

Figure out how to handle the incoming message 291 and update the MQTT state of the camera somehow (what I really want is to use that piece of data to trigger ANOTHER one of these lights)
Add the keepalive

QuantumEntangledAndy · 2023-05-19T04:04:07Z

How did it go. I'd like to pick this up but would need some help since you have the actual device

QuantumEntangledAndy · 2023-05-19T06:16:51Z

I added the callback so we can listen for the floodlight events on 291

I also added the LinkType pings

I really hope that works for you

QuantumEntangledAndy · 2023-05-19T06:17:47Z

Floodlight on/off are reported to the status/floodlight mqtt topic

kevin-david · 2023-05-20T03:49:52Z

Awesome! Pulled it and tried it out. Two discoveries so far:

MQTT names can no longer contain spaces, at least with mosquito on Home Assistant. The error is cryptic but the change I just pushed makes it slightly better
the blocking_send in listen_on_flightlight is causing an error:

thread 'tokio-runtime-worker' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.', 

/Users/kevin/dev/neolink/crates/core/src/bc_protocol/floodlight_status.rs:31:32

I've tried a couple different things so far to fix it but I am way out of my depth with Rust...

QuantumEntangledAndy · 2023-05-24T05:30:40Z

So it was a little difficult but I swapped it to using async callbacks so that we can call send().await inside the callback. Please have another try

The children are `FloodlightStatus`, so that's what we should `yaserde(rename)` to

kevin-david · 2023-05-24T12:39:02Z

src/mqtt/event_cam.rs

@@ -297,6 +372,30 @@ impl<'a> MessageHandler<'a> {
                                "OK".to_string()
                            }
                        }
+                        Messages::FloodlightOn => {
+                            let res = self.camera.set_floodlight_manual(true, 180).await;


Future enhancement: take an optional time here?

kevin-david · 2023-05-24T12:45:39Z

That definitely looks tricky! T: 'static + Send + Sync + for<'a> Fn(&'a Bc) -> BoxFuture<'a, Option<Bc>>,1 - whew. Explains why I couldn't figure it out 😄

I made one last change while debugging - I set up the XML deserialization wrong, but it is working now.

Two future potential changes - curious if you're OK with either of these and have a preferred approach? I'm happy to take another look at either of these for a future PR:

I left one comment where there's a hardcoded value - which AFAIK is the floodlight/camera default. Personally the current value is fine for me, but flexibility might be nice for others.
Enabling MQTT discovery for home assistant: https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery

QuantumEntangledAndy · 2023-05-25T13:58:21Z

I'm quite curious about mqtt discovery. That seems worth pursuing.

QuantumEntangledAndy · 2023-05-25T13:59:19Z

Seems ok. LGTM

kevin-david · 2023-05-25T14:05:16Z

Awesome, thanks for all the help on this! I'll take a look at MQTT discovery next and send a PR once I've got something going. Trying to use this as an excuse to better learn Rust anyway 😄

QuantumEntangledAndy · 2023-05-25T15:37:08Z

If you need any rust pointers let me know. Neolink was my first rust project too

QuantumEntangledAndy · 2023-05-25T15:41:19Z

Also I meant to ask. When you login with debug log enabled there should be a message about the abilities the camera supports. Can you send that over? It would be nice to compare that to a camera. Could maybe auto detect flood light features and when camera is not possible.

kevin-david · 2023-05-26T02:25:06Z

@QuantumEntangledAndy Sure! This is for two different devices, the floodlight and the Duo PoE camera with floodlights as well. I think the second one below is the floodlight without a camera given the response.

<?xml version="1.0" encoding="utf-8"?>
<AbilityInfo>
    <userName>admin</userName>
    <system>
        <subModule>
            <abilityValue>general_rw, norm_rw, version_ro, uid_ro, autoReboot_rw, restore_rw, reboot_rw, shutdown_rw, dst_rw, log_ro, output_rw, performance_ro, upgrade_rw, export_rw, import_rw, bootPwd_rw</abilityValue>
        </subModule>
    </system>
    <network>
        <subModule>
            <abilityValue>port_rw, dns_rw, email_rw, ipFilter_rw, localLink_rw, pppoe_rw, upnp_rw, ntp_rw, netStatus_rw, ptop_rw</abilityValue>
        </subModule>
    </network>
    <alarm>
        <subModule>
            <abilityValue>hddFull_rw, hddError_rw, disconnect_rw, ipConflict_rw, rfAlarm_rw</abilityValue>
        </subModule>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>motion_rw, videoLost_rw, hide_rw</abilityValue>
        </subModule>
    </alarm>
    <image>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>ispBasic_rw</abilityValue>
        </subModule>
    </image>
    <video>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>osdName_rw, osdTime_rw, shelter_rw</abilityValue>
        </subModule>
    </video>
    <replay>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>replay_rw, seek_rw</abilityValue>
        </subModule>
    </replay>
    <PTZ>
        <subModule>
            <abilityValue>control_rw, preset_rw, cruise_rw, track_rw, decoder_rw, ptzInfo_ro</abilityValue>
        </subModule>
    </PTZ>
    <streaming>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>preview_rw, compress_rw, snap_rw, rtsp_rw, streamTable_ro</abilityValue>
        </subModule>
    </streaming>
</AbilityInfo>

<AbilityInfo>
    <userName>admin</userName>
    <system>
        <subModule>
            <abilityValue>general_rw, norm_rw, version_ro, uid_ro, autoReboot_rw, restore_rw, reboot_rw, shutdown_rw, dst_rw, log_ro, performance_ro, upgrade_rw, export_rw, import_rw, bootPwd_rw</abilityValue>
        </subModule>
    </system>
    <network>
        <subModule>
            <abilityValue>port_rw, dns_rw, email_rw, ftp_rw, ftpSchedule_rw, ipFilter_rw, localLink_rw, pppoe_rw, upnp_rw, ntp_rw, netStatus_rw, ptop_rw, autontp_rw</abilityValue>
        </subModule>
    </network>
    <alarm>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>motion_rw</abilityValue>
        </subModule>
    </alarm>
    <image>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>ispBasic_rw, ispAdvance_rw, ledState_rw</abilityValue>
        </subModule>
    </image>
    <video>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>osdName_rw, osdTime_rw, shelter_rw</abilityValue>
        </subModule>
    </video>
    <security>
        <subModule>
            <abilityValue>user_rw, userOnline_rw, bootPwd_rw</abilityValue>
        </subModule>
    </security>
    <PTZ>
        <subModule>
            <abilityValue>control_rw, preset_rw, cruise_rw, track_rw, decoder_rw, ptzInfo_ro</abilityValue>
        </subModule>
    </PTZ>
    <streaming>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>preview_rw, compress_rw, snap_rw, rtsp_rw, streamTable_ro</abilityValue>
        </subModule>
    </streaming>
</AbilityInfo>

Adds config options to enable MQTT Discovery as documented here: https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery As discussed in QuantumEntangledAndy#14 (comment) Unfortinuately, the floodlight/spotlight doesn't show up in the `<AbilityInfo>` returned by the camera: QuantumEntangledAndy#14 (comment) Open to other suggestions on how to detect or enable these options.

kevin-david · 2023-05-27T17:13:07Z

I'm quite curious about mqtt discovery. That seems worth pursuing.

Took a first attempt in #95!

Adds config options to enable MQTT Discovery as documented here: https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery As discussed in QuantumEntangledAndy#14 (comment) Unfortinuately, the floodlight/spotlight doesn't show up in the `<AbilityInfo>` returned by the camera: QuantumEntangledAndy#14 (comment) Open to other suggestions on how to detect or enable these options.

kevin-david mentioned this pull request Dec 11, 2022

Improve MQTT documentation, example config #13

Merged

QuantumEntangledAndy reviewed Dec 12, 2022

View reviewed changes

crates/core/src/bc_protocol/floodlight_status.rs Outdated Show resolved Hide resolved

WIP: Add support for floodlight control over MQTT

1f8296d

kevin-david force-pushed the flood_mqtt branch from 455a30a to 1f8296d Compare January 25, 2023 03:38

Satisify rustfmt

8cbaf06

Remove .vscode, improve logging

6be7ac2

Remove incorrect code for now

355cd48

kevin-david mentioned this pull request Apr 12, 2023

Support devices without HTTP API (such as Reolink Floodlight) starkillerOG/reolink_aio#20

Open

kevin-david mentioned this pull request Apr 24, 2023

Support for controlling Reolink Floodlight thirtythreeforty/neolink#324

Open

Add floodlight status and keep alive pings

be10aae

fmt

1824e06

Improve MQTT error message

d90f0b1

Async handle_msg closures

864f58d

QuantumEntangledAndy and others added 2 commits May 24, 2023 12:33

Fmt

f997552

Fix XML deserialization of FloodlightStatusList

5f96b2e

The children are `FloodlightStatus`, so that's what we should `yaserde(rename)` to

kevin-david changed the title ~~WIP: Add support for floodlight control over MQTT~~ Add support for floodlight control over MQTT May 24, 2023

kevin-david commented May 24, 2023

View reviewed changes

kevin-david marked this pull request as ready for review May 24, 2023 12:46

QuantumEntangledAndy merged commit fdff244 into QuantumEntangledAndy:master May 25, 2023

kevin-david deleted the flood_mqtt branch May 25, 2023 14:05

kevin-david mentioned this pull request May 27, 2023

Feature/optional MQTT discovery #95

Merged

QuantumEntangledAndy mentioned this pull request Jun 6, 2023

Camera connection lost in mqtt command #97

Closed

jakommo mentioned this pull request Nov 15, 2023

mqtt control topic missing for poe floodlight #189

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for floodlight control over MQTT #14

Add support for floodlight control over MQTT #14

kevin-david commented Dec 11, 2022

QuantumEntangledAndy commented Dec 12, 2022

QuantumEntangledAndy commented Dec 12, 2022

QuantumEntangledAndy commented Dec 13, 2022

kevin-david commented Dec 13, 2022 •

edited

Loading

QuantumEntangledAndy commented Dec 13, 2022 •

edited

Loading

QuantumEntangledAndy commented Dec 14, 2022

kevin-david commented Dec 14, 2022 •

edited

Loading

QuantumEntangledAndy commented Dec 14, 2022

QuantumEntangledAndy commented Jan 5, 2023

kevin-david commented Jan 6, 2023 •

edited

Loading

kevin-david commented Jan 25, 2023 •

edited

Loading

QuantumEntangledAndy commented Jan 25, 2023

QuantumEntangledAndy commented Jan 25, 2023

QuantumEntangledAndy commented Jan 25, 2023

kevin-david commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023 •

edited

Loading

kevin-david commented Jan 26, 2023 •

edited

Loading

kevin-david commented Jan 26, 2023

QuantumEntangledAndy commented May 19, 2023

QuantumEntangledAndy commented May 19, 2023

QuantumEntangledAndy commented May 19, 2023

kevin-david commented May 20, 2023 •

edited

Loading

QuantumEntangledAndy commented May 24, 2023

kevin-david May 24, 2023

kevin-david commented May 24, 2023 •

edited

Loading

QuantumEntangledAndy commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

kevin-david commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

kevin-david commented May 26, 2023

kevin-david commented May 27, 2023

Add support for floodlight control over MQTT #14

Add support for floodlight control over MQTT #14

Conversation

kevin-david commented Dec 11, 2022

QuantumEntangledAndy commented Dec 12, 2022

QuantumEntangledAndy commented Dec 12, 2022

QuantumEntangledAndy commented Dec 13, 2022

kevin-david commented Dec 13, 2022 • edited Loading

QuantumEntangledAndy commented Dec 13, 2022 • edited Loading

QuantumEntangledAndy commented Dec 14, 2022

kevin-david commented Dec 14, 2022 • edited Loading

QuantumEntangledAndy commented Dec 14, 2022

QuantumEntangledAndy commented Jan 5, 2023

kevin-david commented Jan 6, 2023 • edited Loading

kevin-david commented Jan 25, 2023 • edited Loading

QuantumEntangledAndy commented Jan 25, 2023

QuantumEntangledAndy commented Jan 25, 2023

QuantumEntangledAndy commented Jan 25, 2023

kevin-david commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023

QuantumEntangledAndy commented Jan 26, 2023 • edited Loading

kevin-david commented Jan 26, 2023 • edited Loading

kevin-david commented Jan 26, 2023

QuantumEntangledAndy commented May 19, 2023

QuantumEntangledAndy commented May 19, 2023

QuantumEntangledAndy commented May 19, 2023

kevin-david commented May 20, 2023 • edited Loading

QuantumEntangledAndy commented May 24, 2023

kevin-david May 24, 2023

Choose a reason for hiding this comment

kevin-david commented May 24, 2023 • edited Loading

QuantumEntangledAndy commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

kevin-david commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

QuantumEntangledAndy commented May 25, 2023

kevin-david commented May 26, 2023

kevin-david commented May 27, 2023

kevin-david commented Dec 13, 2022 •

edited

Loading

QuantumEntangledAndy commented Dec 13, 2022 •

edited

Loading

kevin-david commented Dec 14, 2022 •

edited

Loading

kevin-david commented Jan 6, 2023 •

edited

Loading

kevin-david commented Jan 25, 2023 •

edited

Loading

QuantumEntangledAndy commented Jan 26, 2023 •

edited

Loading

kevin-david commented Jan 26, 2023 •

edited

Loading

kevin-david commented May 20, 2023 •

edited

Loading

kevin-david commented May 24, 2023 •

edited

Loading