2-way audio with ICSee/dvrip cameras

[felipecrs]

Many cheap chinese cameras are using this ICSee. I bought this one: https://aliexpress.com/item/1005005731238053.html

It's relatively good, although it does not have the capability of changing the stream codec from H265 to H264.

But I have another ICSee based camera as well.

The app is good as well. A lot better than Tuya.

It looks like these cameras uses the DVRIP protocol. My RTSP URLs looks like:

rtsp://192.168.1.21:554/user=tema_password=<password>_channel=0_stream=0&onvif=0.sdp

It would be awesome if 2-way audio could work with it.

[felipecrs]

2-way audio works through its web UI:

Although it requires the installation of a "VideoPlayTool" to properly open it.

@AlexxIT if someday you are interested in implementing this I can give you direct access to the camera.

[felipecrs]

Oh, and by the way DVRIP source also works with this camera:

streams:
  icsee_camera:
    - dvrip://tema:<password>@192.168.1.21:34567?channel=0&subtype=1

Meaning that maybe the support for 2-way audio could be general for any DVRIP-based camera.

[slydiman]

Agreed. It would be great to support 2-way audio for DVRIP/XM/Sofia-based cameras.

[Djeloan]

Oh yes! I was trying to do it but couldn't ... i can talk trough that xmeye application tho.

[slydiman]

Oh yes!

@Djeloan, Please vote on the initial message at the top of this page. The author must see that it is important enhancement.

[slydiman]

Here is the Talk command implementation for DVRIP protocol
https://github.com/667bdrm/sofiactl/blob/master/sofiactl.pl#L2286
The source file must be PCM G711 ALAW 8kHz mono.

    $decoded = $dvr->PrepareGenericCommand(IPcam::TALK_CLAIM, { # 1434
        Name => 'OPTalk',
        OPTalk => {
            Action => "Claim",
            AudioFormat => {
                BitRate => 128,
                EncodeType => "G711_ALAW",
                SampleBit => 8,
                SampleRate => 8000
            }
        }
    });

    $decoded = $dvr->PrepareGenericCommand(IPcam::TALK_REQ, { # 1430
        Name => 'OPTalk',
        OPTalk => {
            Action => "Start",
            AudioFormat => {
                BitRate => 128,
                EncodeType => "G711_ALAW",
                SampleBit => 8,
                SampleRate => 8000
            }
        }
    });

    if ($cfgInputFile ne '') {
        my $size = -s $cfgInputFile;
        print "File size: $size\n";
        my $count = 0;
        open(IN, "< $cfgInputFile");
        while ($count < $size) {
            my $data;
            read(IN, $data, 320);
            #print $data . "\n";
            $count += 320;
            my $audio_pkt = $dvr->BuildRawPacket(IPcam::TALK_CU_PU_DATA, # 1432
                pack('CCCCCCS', (0x00, 0x00, 0x01, 0xFA, 0x0E, 0x02, 320)) . $data);  # 320 bytes 40ms audio to camera
            $dvr->{socket}->send($audio_pkt);

            my $reply_head = $dvr->GetReplyHead();  # 1433
            $dvr->GetReplyData($reply_head);  # 8+160 bytes data - 20ms mic audio from camera
        }
        close(IN);
    }

[AlexxIT]

Supported in latest master:

streams:
  stream: dvrip://username:password@192.168.1.123:34567
  tts_only: dvrip://username:password@192.168.1.123:34567?backchannel=1
  two_way:
    - dvrip://username:password@192.168.1.123:34567
    - dvrip://username:password@192.168.1.123:34567?backchannel=1

[felipecrs]

OMG this is amazing. I will test it asap.

[Djeloan]

Supported in latest master:

Means that will be available in some next update or the current one (1.7.1) ?
Tested it now, but couldn't get it to work, so im guessing it's in the next update

[felipecrs]

https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#go2rtc-dev-version

[AlexxIT]

@felipecrs have you tried?

[slydiman]

Great work! Thank you very much!
But it works with the microphone only from go2rtc addon UI: Links, select video+audio+microphone = two way audio from camera, click webrtc.html local WebRTC viewer.
I was shocked - there is no way to turn on the microphone in the HA UI. The solution with browser_mod does not work for me (in case of few HA users and Android app).
The next step is to ubdate WebRTC camera custom card to enable the microphone.
Install the Frigate/Frigate addon + Frigate integration + Frigate custom card for 1 button microphone is overkill!

[felipecrs]

Frigate Card is now supporting external go2rtc in latest dev. Meaning you don't need Frigate.

[felipecrs]

@felipecrs have you tried?

I'm sorry not yet. But I will try it very soon. My ICSee camera is currently far away from my current location.

[AlexxIT]

@felipecrs you can send audio to it an check result via stream (if you have remote access)

[felipecrs]

Oh, yes. This is what I see:

{
  "producers": [
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/8aa14331e4a4907604463ca9c8ca54eb",
      "remote_addr": "127.0.0.1:38036",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.7.1\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QMKawsqAoBbpp4AAADAAgAAAMAxHhAIVA=,aO44sA==; profile-level-id=640C29\r\na=control:streamid=0\r\n",
      "medias": ["video, recvonly, H.264 High 4.1"],
      "receivers": ["96 H264, bytes=5876964, senders=1"],
      "recv": 5928476
    },
    {
      "type": "RTSP active producer",
      "url": "rtsp://127.0.0.1:8554/rua_h265?audio=all",
      "remote_addr": "127.0.0.1:8554",
      "user_agent": "go2rtc/1.7.1",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.7.1\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 PCMA/8000\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/8000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=158856E500\r\na=control:trackID=1\r\n",
      "medias": [
        "audio, recvonly, PCMA/8000",
        "audio, recvonly, MPEG4-GENERIC/8000"
      ],
      "receivers": ["96 PCMA/8000, bytes=297920, senders=1"],
      "recv": 453571
    }
  ],
  "consumers": [
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 srflx 179.106.189.52:64259 related 192.168.1.15:64259",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000"
      ],
      "receivers": ["111 OPUS/48000/2, bytes=16351, senders=0"],
      "senders": [
        "102 H264, bytes=1290962, receivers=1",
        "8 PCMA/8000, bytes=65920, receivers=1"
      ],
      "recv": 23261,
      "send": 1371330
    }
  ]
}

[felipecrs]

Oh sorry, wait a sec. I think I made a mistake.

[AlexxIT]

I don't need logs. Just check via live stream if you can hear some audio file. You can send it via WebUI links.html page

[felipecrs]

Yes I can hear myself!!! That's amazing! It's working nice, thanks a lot!

[AlexxIT]

Nice. Thanks to @slydiman for perl and sharing access to camera.

[felipecrs]

go2rtc.zip (latest artifact for amd64)

If someone wants to quickly try this out with Frigate:

ha addons stop ccab4aaf_frigate
wget https://github.com/AlexxIT/go2rtc/files/12884449/go2rtc.zip -O /config/go2rtc
chmod +x /config/go2rtc
ha addons start ccab4aaf_frigate

[slydiman]

Frigate Card is now supporting external go2rtc in latest dev. Meaning you don't need Frigate.

I tried it yesterday - no luck. @felipecrs, can you provide a link to the documentation or config example?

[felipecrs]

@slydiman, before anything else you need to have go2rtc exposed as https with a valid certificate. I do that using the NGINX Proxy Manager add-on.

But go2rtc itself also supports https/tls, although it will be harder to configure.

Do you have this part done?

[slydiman]

Sure. HA with https and go2rts master work just fine.

I tried to update AlexxIT's WebRTC camera custom card today. I have successfully enabled the microphone recording in the browser but stuck with creating the correct rtc offer request.

[felipecrs]

Sorry, I have no config samples to provide. As I mentioned, I use NGINX Proxy Manager to expose go2rtc through HTTPS. Also, I don't use the Frigate card without the Frigate integration.

Frigate documentation says that 2 way audio requires a frigate camera provided by the Frigate integration.

I think you missed my point that this feature was only added to the development branch of the Frigate card.

Maybe you can find more help in dermotduffy/frigate-hass-card#1206.

[slydiman]

I have configured NGINX to access go2rtc addon through HTTPS.
Finally I was able to get 2-way audio working with the latest dev frigate-card and the following config

        - type: custom:frigate-card
          cameras:
          - live_provider: go2rtc
            go2rtc:
              url: https://mydomain:1985
              stream: cam_with_audio
              modes:
               - webrtc
          menu:
            buttons:
              microphone:
                enabled: true

But go2rtc does not have any authorization and it will be a security hole. I can't use it outside my home network.
The bottom line is that we still need to update WebRTC Camera card to support microphone with go2rtc through hassio ingress.

[felipecrs]

But go2rtc does not have any authorization and it will be a security hole. I can't use it outside my home network.

Yes, that is far from ideal. But you can set go2rtc to require user and password, and maybe you can embed the user and password in the go2rtc.url in the frigate card like https://user:password@mydomain.com:1935.

I agree it would be great if WebRTC Camera had built-in support for 2-way audio. But @AlexxIT is aware, he just didn't have the time and to ship it yet.

Similarly, support for go2rtc without its direct exposure is something @dermotduffy is also aware and he would like to ship it. Likewise, he probably didn't have time to work on it yet.

Needless to say that both has no obligation to work on anything and probably contributions are welcome.

Few months ago we did not have any kind of 2-way audio support at all in Home Assistant. Things are evolving, and will continue to evolve and be more seamless as the time goes. :)

[slydiman]

I have updated WebRTC Camera to enable the microphone for 2-way audio.

AlexxIT/WebRTC#579

It works just fine in a browser with go2rtc addon. HA must be HTTPS.

Unfortunately the mic does not work inside the Android HA app - it shows that the mic is used, but there is no any transmission.

[AlexxIT]

https://github.com/AlexxIT/go2rtc/releases/tag/v1.8.0

[Cr4z33]

Sorry if my question might sound silly, but is there any chance this can work also with a Amcrest AD410 video doorbell or does it have no DVRIP support?

[slydiman]

Amcrest AD410 video doorbell

Seems it uses VoIP/SIP protocols for audio conversations. It is not DVRIP compatible at all.

[Cr4z33]

@slydiman that's what I was fearing... OK thanks.

[AlexxIT]

Amcrest AD410 doorbell support RTSP with two way audio

[felipecrs]

Guys, since we are talking about two-way audio here, I have two feature requests in Frigate Card that may interest you too:

• Add some mechanism to automatically unmute inbound audio when using the microphone dermotduffy/frigate-hass-card#1130
• Allow to trigger card automations on microphone unmuted dermotduffy/frigate-hass-card#1241

If you find them useful, please vote on them with a thumbs up. But please do not vote if you don't find them useful.

[felipecrs]

Another one:

• Develop a general purpose proxy integration dermotduffy/frigate-hass-card#1299

[PaulBiod]

Hi,
I'm also using a chinese icsee cam which supports 2 ways audio, but I can't get it to work (updated to last webrtc.), I have no microphone on the stream.

My card looks like this :

any help would be appreciated

[felipecrs]

How does the go2rtc config looks like?

Also is your HA exposed as HTTPS?

[PaulBiod]

I use webrtc to show cards of my onvif cams (setup in ICSee)

Yes, my HA is exposed as https via duckdns

[felipecrs]

You need to add a go2rtc configuration with the drip: back channel link.

[PaulBiod]

sorry for beeing noob, but how can I achieve this ? I only installed webrtc to have greater cam cards, and go2rtc is included in web2rtc hacs integration

[felipecrs]

I use the go2rtc add-on, but I think you can probably create one file named go2rtc.yaml at config with something like:

streams:
  camera.jardin_profile_000: 
    - "{input}"
    - dvrip://username:password@192.168.1.123:34567?backchannel=1

But I am not sure if the "{input}" line will work as I intended. @AlexxIT can comment more on this.

[slydiman]

go2rtc add-on config must look like

streams:
  camera.jardin_profile_000: 
   - dvrip://admin:password@192.168.1.123:34567?channel=0&subtype=1
   - dvrip://admin:password@192.168.1.123:34567?backchannel=1

My card looks like this :

Here is the correct sample of webrtc-camera card AlexxIT/WebRTC#579 (comment)
Forget about mic: true. It was my suggestion in the pool request. The final implementation requires media: video,audio,microphone.
In your case

type: 'custom:webrtc-camera'
ui: true
streams:
- url: camera.jardin_profile_000
  mode: webrtc
  name: '🎙️ Mic on'
  media: video,audio,microphone
muted: false
...

[PaulBiod]

Hi all,
thanks for your time, so, If I understood, I just have to create a file named go2rtc.yaml in the same path where my HAOS config file is, with just these lines (with my IPs of course and credentials):

streams:
camera.jardin_profile_000:

• dvrip://admin:password@192.168.1.123:34567?channel=0&subtype=1
• dvrip://admin:password@192.168.1.123:34567?backchannel=1

And then use this code for the card :

type: 'custom:webrtc-camera'
ui: true
streams:

• url: camera.jardin_profile_000
  mode: webrtc
  name: '🎙️ Mic on'
  media: video,audio,microphone
  muted: false
  ...

Thanks guys

[slydiman]

If I understood, I just have to create a file named go2rtc.yaml in the same path where my HAOS config file is

Right. Be sure the addon go2rtc is started. You can open go2rtc addon settings and click Open web UI or enable the option Show in sidebar, then click on go2rtc in left sidebar. You will see go2rtc's web UI. Then you can click on Config tab, edit the config and then click the button Save & Restart.

Note the yaml rules. The padding for streams: and muted: false must be the same. It is important.

streams:
 - url: camera.jardin_profile_000
   mode: webrtc
   name: '🎙️ Mic on'
   media: video,audio,microphone
muted: false

... at the end means etc. Don't copy ... as is ;)

[PaulBiod]

Ok guys,
so, I created the file :

Then, restarted HAOS, went to my dashboard, and create a card :

The result is the same (no micro) :

Here is my onvifer result showing the ports :

Any idea ?
Thanks

[slydiman]

No need to restart HAOS. It is enough to restart only go2rtc addon after changing go2rtc.yaml

If you need to see the name of the stream with the mic icon and be able to switch off the mic just add a second stream with the same camera, other name and media: video,audio

Be sure you have set permissions to the browser to use the microphone. You must see the rec icon on the page tab. If you are using Google Chrome
Video with muted audio:

Video and audio:

Video, audio and microphone:

Click on the mic icon in the address bar, allow the mic access and select the audio input (microphone) device:

[PaulBiod]

Thanks, indeed the micro in the adress bar was disabled, I enabled it, refresh the page, put a second stream, now I can switch between the streams, but the stream with micro ON doesn't send any audio from my PC...

[PaulBiod]

So, I tried with my tablet, it asked me for audio permissions, then I can hear myself speaking, but very low, I only realised it because of the larsen (being close to the camera). The cam speaker is at 100% volume, when I try with icsee, it's clear and loud.
With my laptop, no sound at all. Tried with chrome and edge, asked me for micro permission, refreshed, still no sound...

[eerison]

I didn't test it yet, But I want to say thank you guys for all effort in this feature ❤️

[eerison]

Hey @dbuezas
I guess you was trying to make this work, don't you?

[felipecrs]

It's working just fine for me since the day it was implemented.