Two way audio #243

pergolafabio · 2022-01-21T08:49:21Z

Hi @AlexxIT

Just a question, but is there a possibility to have two way audio enabled?

Thnx in advance

danielducu · 2022-01-21T13:46:20Z

I would definitely pay for that :)

pergolafabio · 2022-01-21T13:47:52Z

Yes, latest android app (beta) supports now microphone and speaker

rbray89 · 2022-01-26T14:34:39Z

Technically this *may" be feasible, but it would likely depend on the method of backend audio. Either ONVIF 2-way would be assumed, or something like SIP would have to be used. Currently Home assistant is not aware of the audio sink, so it would have to be configured entirely in the component.

pergolafabio · 2022-01-26T14:36:51Z

Mine cameras are based on ISAPI, to start twoway audio, i need to send 2 commands (no onvif)
Synology is also using the same method for my hikvision door camera

would be a great feature, so we can chat with the HA app :-)

rbray89 · 2022-01-26T17:14:41Z

WebRTC is also limited in supported codecs, which is why audio on some cameras doesn't work. This won't be an easy change to get it to work on devices that support 2way audio.

pergolafabio · 2022-01-26T17:20:25Z

Is it worth a try? Would be great!!
Mine audio codec is ulaw, it's the default audio protocol, most common used... One way audio already works now with current card...
But I have no idea how to enable the mic :-) and to send the ISAPi command from the card to active it on the stream...

pergolafabio · 2022-01-26T17:21:44Z

I have a trace how Synology is doing it, they just send 2 command to active the two-way audio

AlexxIT · 2022-02-06T16:14:25Z

RTSP is spec for camera stream translation. There is no spec for two way audio. Each company use their own tech. Usual it's some kind of P2P.

pergolafabio · 2022-02-06T16:21:32Z

yeah, mine does work with sending 2 commands, do you think its easy to integrate?

rbray89 · 2022-02-06T21:36:23Z

@danielducu I don't think you understand the difficulty here.

vendors all do this differently, and it's not documented, so everything has to be reverse engineered.
webRTC only uses a few different audio codecs, so they'd almost certainly have to transcode things on the server.

nanosonde · 2022-05-09T09:34:58Z

If we rely on standards-based cameras, it could work though.

ONVIF Profile T: Chapter 8.12 Audio Output streaming: https://www.onvif.org/wp-content/uploads/2018/09/ONVIF_Profile_T_Specification_v1-0.pdf
More practical details: https://www.happytimesoft.com/knowledge/audio-back-channel.html
So basically a third stream towards the camera (besides video and audio from camera) is negotiated in the SDP of the server.
The audio out stream then just flows in the opposite direction of the other two RTP streams (sendonly vs. recvonly).

Video doorbell cameras mostly rely on two-way audio negotiated via SIP.
It did a PoC implemenation for Homebridge here: homebridge-plugins/homebridge-camera-ffmpeg#1253
Ok, this is not related to WebRTC directly.

Solutions for propriertary vendor solutions could be doable. Example neolink for Reolink cameras: thirtythreeforty/neolink#268

Concerning ISAPI I only found this: https://www.hikvision.com/content/dam/hikvision/vn/webinar/Thang3_Hikvision_Tich_Hop_He_Thong_Overview-of-3rd-Party-Integration.pdf

pergolafabio · 2022-05-09T12:16:15Z

I have the Hikvision, and also all ISAPI documentations, so if i can help
i also know the ISAPI urls that should be used to activcate twoway audio

mattkerrison · 2022-07-19T05:23:15Z

The Amcrest AD410 Doorbell supports ONVIF Profile T for 2-way audio.

The Dahua integration for Home Assistant hasn't been able to get it working yet but the dream would be WebRTC with 2 way audio, could finally get rid of the garbage Amcrest app

AlexxIT · 2022-07-22T16:07:52Z

Do you know software for any OS, that can use ONVIF Profile T for 2 way audio connection?

conorlap · 2022-07-22T16:15:44Z

@AlexxIT
tinyCam on Android can do 2 way audio with these settings:

I found this out via Reddit: https://www.reddit.com/r/BlueIris/comments/lpkt8z/achieved_local_doorbell_with_2_way_audio/

AlexxIT · 2022-07-22T17:23:08Z

Seems like support backchannel in RTSP is not a big problem. Now I need to find some chip camera with this function in a local store.

AlexxIT · 2022-07-22T19:39:23Z

I'm wrote little app for test camera answer about backchannel. Can you test it?
https://github.com/AlexxIT/rtsplog

pergolafabio · 2022-07-22T19:44:27Z

That's a test for onvif? I have some foscam C1 cameras that have two way audio, maybe I can test too

AlexxIT · 2022-07-22T20:00:07Z

That test for any RTSP camera with 2 way audio. I'm already tested Sonoff camera and one Hikvision doorbell without success about backchannel in answer.

pergolafabio · 2022-07-22T20:02:16Z

Indeed ,I have hikvision too, for hikvision you need to send an ISAPI command to activate two way audio, I have the commands , also a wireshark trace from my Synology surveillance centre, where two-way audio is supported

AlexxIT · 2022-07-22T20:12:21Z

As I can see in this document - application just need to send right header with RTSP request. Can you see it in trace?

pergolafabio · 2022-07-22T20:15:28Z

No , Synology uses ISAPI on hikvision doorbell cameras , not rtsp , also onvif not supported, it's not the correct profile...
I also have foscam cameras, I can test later your tool

mattkerrison · 2022-07-23T02:12:12Z

I'm wrote little app for test camera answer about backchannel. Can you test it?

https://github.com/AlexxIT/rtsplog

Will try tomorrow when I have access to the AD410 camera

pergolafabio · 2022-07-23T11:02:28Z

Hey @AlexxIT , these 2 commands are fired with digest auth, thats what happens when i press the MIC button on the synology software....

conorlap · 2022-07-23T12:47:26Z

I'm wrote little app for test camera answer about backchannel. Can you test it? https://github.com/AlexxIT/rtsplog

I can test in the next day or two, how do I execute the rtsplog_amd64 file on Debian?

AlexxIT · 2022-07-24T04:43:43Z

@pergolafabio For now I don't have time for support some proprietary technology.

RTSP backchannel looks like nice standard. Which are not hard to support. But I can't find any cameras that support it yet.

pergolafabio · 2022-07-24T06:53:40Z

@pergolafabio For now I don't have time for support some proprietary technology.

RTSP backchannel looks like nice standard. Which are not hard to support. But I can't find any cameras that support it yet.

Ok, np...

And what about onvif?

AlexxIT · 2022-07-24T07:51:45Z

As I can understand, ONVIF just helps application to get right RTSP link. Plus some PTZ commands. I could be wrong.

AlexxIT · 2022-07-30T10:33:04Z

No. I don't have any camera that support ONVIF. Hope Dahua IPC-K42/22 will be first. But I can't order it without backchannel confirmation.

pergolafabio · 2022-07-30T12:48:10Z

Going to try again later, I changed my rtsp port , going to change back to 554

AlexxIT · 2022-08-04T12:44:02Z

Well. Good news. I just bought Dahua IPC-K42 (wi-fi version). And it has RTSP backchannel.
I'll try to add support in the next big update. I can't promise it will be soon.

pergolafabio · 2022-08-04T12:55:54Z

Nice to hear, maybe my foscams will work too :-)

AlexxIT · 2022-08-21T08:13:31Z

@mattkerrison you can check 2-way audio support:

https://community.home-assistant.io/t/realtime-camera-streaming-without-any-delay-webrtc/258216/420?u=alexxit

https://github.com/AlexxIT/go2rtc

pergolafabio · 2022-08-21T08:28:20Z

Nice, gonna check foscam too

pergolafabio · 2022-08-22T13:04:39Z

was testing this go2rtc, when i open web interface , i can access the cameras and view then in the addon web-ui
i added some rtsp links , works fine, all local , on http
https is for only avaible from outside, using nginx , i also added for test port forward 8554 and 1984 , but then i dont see the stream ... will this not work with nginx?

AlexxIT · 2022-08-22T13:53:08Z

Oh. You shouldn't forward RTSP port outside. Also you shouldn't forward API port, because this is insecure. Anyone can see your streams without auth...
Read docs carefully.

pergolafabio · 2022-08-22T13:57:49Z

yeah, it was just for testing, already closed
the problem is that i cant access my HA instance on https locally , only external
local i can only access http

AlexxIT · 2022-08-22T14:00:02Z

This is your setup problem. I have HTTPS inside and outside. This is fixed by Hairpin NAT or local DNS on router...

pergolafabio · 2022-08-22T14:01:52Z

ok, will have a look next time
propoably my foscams were not supported anyway, i already posted those logs before in this issue thread

AlexxIT · 2022-08-22T14:04:04Z

Also microphone will work on localhost. You can run go2rtc on your PC/laptop for tests.

Maybe some browsers has disable flag for this restriction. I don't know.

pergolafabio · 2022-08-22T14:05:51Z

seems defender is blocking the binary :-)

pergolafabio · 2022-08-22T14:06:35Z

Trojan:Win32/Trickbot!ml :-)

pergolafabio · 2022-08-22T14:09:52Z

ok, with binary it works, tested some rtsp , but no "mic" buton is present, so probably out of luck
or should i see the mic button?

AlexxIT · 2022-08-22T14:23:00Z

False virus detection because packer:
upx/upx#337
upx/upx#437

There is no warnings for unpacked version. But it is 15MB+.
I'll think what can do with this. Thanks for the point.

And stop generate so many separate messages.

JxxIT · 2022-12-11T16:32:54Z

Is there any update on this? Cause i really like to see this feature 😀.

felipecrs · 2023-01-12T00:19:33Z

I think this can finally become a thing now that the integration leverages go2rtc. :)

danielducu · 2023-01-12T08:27:39Z

I think this can finally become a thing now that the integration leverages go2rtc. :)

Not much, you can only do it with Onvif profile T cameras so not much people will benefit

AlexxIT · 2023-01-12T08:51:41Z

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

danielducu · 2023-01-12T09:03:36Z

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

Thanks a lot for your work man! don't get me wrong.. I'm a happy go2rtc user hehe ... this has come a long way so please, keep up the good work hehe !

danielducu · 2023-01-12T09:04:09Z

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

You're awesome! :)

felipecrs · 2023-01-12T12:33:29Z

I think this can finally become a thing now that the integration leverages go2rtc. :)

Not much, you can only do it with Onvif profile T cameras so not much people will benefit

That's already enough. I hope support for profile T gets someday added to OpenIPC thus anyone with any camera can benefit.

nanosonde · 2023-02-24T15:55:42Z

Hi there!

Maybe you find my new project useful: https://github.com/nanosonde/sip2rtsp

It is docker container that uses baresip, pulseaudio and gstreamer RTSP server to provide a solution that connects to a SIP peer (audio only) and a video camera to present an ONVIF profile T video doorbell with two-way audio support.

There is also a block diagram which shows how the components interact.

pergolafabio · 2023-02-28T20:41:53Z

AlexxIT/go2rtc@5846cbd

thnx!!

barrelltitor · 2023-05-21T11:10:52Z

Is two way audio possible with eufy pan tilt 2k?

AlexxIT added the enhancement New feature or request label Feb 6, 2022

AlexxIT mentioned this issue Apr 10, 2022

2 way audio request #279

Closed

AlexxIT mentioned this issue Jul 30, 2022

Support for IPC-K42 (Imou Cube 4MP) rroller/dahua#35

Closed

conorlap mentioned this issue Aug 1, 2022

Possibility to talk with doorbell rroller/dahua#181

Open

AlexxIT added the go2rtc label Aug 21, 2022

pergolafabio closed this as completed Feb 28, 2023

Two way audio #243

Two way audio #243

Comments

pergolafabio commented Jan 21, 2022

danielducu commented Jan 21, 2022

pergolafabio commented Jan 21, 2022

rbray89 commented Jan 26, 2022

pergolafabio commented Jan 26, 2022

rbray89 commented Jan 26, 2022

pergolafabio commented Jan 26, 2022

pergolafabio commented Jan 26, 2022

AlexxIT commented Feb 6, 2022 • edited Loading

pergolafabio commented Feb 6, 2022

rbray89 commented Feb 6, 2022

nanosonde commented May 9, 2022

pergolafabio commented May 9, 2022

mattkerrison commented Jul 19, 2022

AlexxIT commented Jul 22, 2022

conorlap commented Jul 22, 2022

AlexxIT commented Jul 22, 2022

AlexxIT commented Jul 22, 2022

pergolafabio commented Jul 22, 2022

AlexxIT commented Jul 22, 2022 • edited Loading

pergolafabio commented Jul 22, 2022

AlexxIT commented Jul 22, 2022

pergolafabio commented Jul 22, 2022

mattkerrison commented Jul 23, 2022

pergolafabio commented Jul 23, 2022

conorlap commented Jul 23, 2022

AlexxIT commented Jul 24, 2022

pergolafabio commented Jul 24, 2022

AlexxIT commented Jul 24, 2022

AlexxIT commented Jul 30, 2022

pergolafabio commented Jul 30, 2022

AlexxIT commented Aug 4, 2022

pergolafabio commented Aug 4, 2022

AlexxIT commented Aug 21, 2022 • edited Loading

pergolafabio commented Aug 21, 2022

pergolafabio commented Aug 22, 2022

AlexxIT commented Aug 22, 2022 • edited Loading

pergolafabio commented Aug 22, 2022

AlexxIT commented Aug 22, 2022

pergolafabio commented Aug 22, 2022

AlexxIT commented Aug 22, 2022 • edited Loading

pergolafabio commented Aug 22, 2022

pergolafabio commented Aug 22, 2022

pergolafabio commented Aug 22, 2022

AlexxIT commented Aug 22, 2022 • edited Loading

JxxIT commented Dec 11, 2022

felipecrs commented Jan 12, 2023

danielducu commented Jan 12, 2023

AlexxIT commented Jan 12, 2023

danielducu commented Jan 12, 2023

danielducu commented Jan 12, 2023

felipecrs commented Jan 12, 2023

nanosonde commented Feb 24, 2023

pergolafabio commented Feb 28, 2023

barrelltitor commented May 21, 2023

AlexxIT commented Feb 6, 2022 •

edited

Loading

AlexxIT commented Jul 22, 2022 •

edited

Loading

AlexxIT commented Aug 21, 2022 •

edited

Loading

AlexxIT commented Aug 22, 2022 •

edited

Loading

AlexxIT commented Aug 22, 2022 •

edited

Loading

AlexxIT commented Aug 22, 2022 •

edited

Loading