When go2rtc self-requests a source capable of backchannel, it breaks backchannel #1133

felipecrs · 2024-05-22T17:12:19Z

It was very tricky to figure out that this was actually caused by go2rtc.

Here is my config:

streams:
  vto:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=1
    # copy PCMA track from vto
    - rtsp://127.0.0.1:8554/vto?audio=pcma

In fact, I have a script to output the stream rather than using it directly because of #49, but I am removing it from this test to simplify things.

Also, my password is leaked but that is ok, I changed already. That's because I could not figure out how to remove it from the info API from go2rtc so it would appear in the video anyway.

Here is the problem:

chrome_tLNTWxaf4t.mp4

Basically, if I open vto first, everything works ok. I can 2-way audio with it normally. Notice the sendonly track.

However, if I open vto_hd first, then I can no more do 2-way audio with the vto stream.

I think go2rtc should attempt to discover again more tracks if 2-way audio is requested. In fact, I don't understand why it only discovers the recvonly tracks when I request vto_hd.

The text was updated successfully, but these errors were encountered:

felipecrs · 2024-05-22T17:18:24Z

I am working around it at the moment with:

streams:
  vto:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=1
    # copy PCMA track from vto
    # - rtsp://127.0.0.1:8554/vto?audio=pcma
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=0#media=audio

TokarevSergey · 2024-05-22T17:38:07Z

Было очень сложно понять, что на самом деле это вызвано go2rtc.

Вот моя конфигурация:
streams:
  vto:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:8EiPuNd6pzjR686_@192.168.1.40/cam/realmonitor?channel=1&subtype=1
    # copy PCMA track from vto
    - rtsp://127.0.0.1:8554/vto?audio=pcma
На самом деле у меня есть скрипт для вывода потока, а не для его прямого использования из-за #49 , но я удаляю его из этого теста, чтобы упростить задачу.

Кроме того, мой пароль утек, но это нормально, я уже сменил. Это потому, что я не смог придумать, как убрать его из info API от go2rtc, чтобы оно все равно появлялось в видео.

Вот проблема:

chrome_tLNTWxaf4t.mp4
В принципе, если сначала открыть vto, все работает нормально. Я могу нормально использовать двустороннюю аудиосвязь. Обратите внимание на трек sendonly.

Однако, если я открою vto_hdсначала, я больше не смогу использовать двустороннюю аудиосвязь с vtoпотоком.

Я думаю, что go2rtc должен попытаться снова обнаружить больше дорожек, если запрошена двусторонняя аудиосвязь. На самом деле, я не понимаю, почему он обнаруживает только треки, когда я запрашиваю vto_hd.

Good afternoon. Tell me, does your sound become quieter and interrupted?

felipecrs · 2024-05-22T17:45:39Z

go2rtc simply doesn't detect the backchannel (sendonly) track, and that's why 2-way audio doesn't work. Whatever other symptom is irrelevant.

felipecrs · 2024-05-22T18:22:41Z

BTW I'm testing this more and even with the workaround, sometimes go2rtc simply "forgets" about the sendonly track. I still can't figure out why. But if I restart go2rtc, then it catches the sendonly track again.

Which makes using the VTO doorbell with go2rtc impossible at this point.

felipecrs · 2024-05-22T18:51:33Z

EDIT: This is off-topic and a different, device-related issue.

Hm... I actually understand now. When someone calls the doorbell, like when someone presses it, the doorbell closes the RTSP stream momentarily. And go2rtc tries to get the connection again but this time it gets without sendonly track. Only if I ask go2rtc to get the connection after the call is finished it will be able to get sendonly again.

felipecrs · 2024-05-22T19:00:46Z

@AlexxIT I don't know how feasible it is, but it could probably be solved if go2rtc would attempt to reload the sources if a consumer is requesting microphone but there is no sendonly track already in the sources.

TokarevSergey · 2024-05-22T19:12:03Z

Yes, yes, I’ve already tried to write this several times.

felipecrs · 2024-05-22T19:17:18Z

Another option is maybe #52, maybe CGI doesn't have the same limitation.

Or maybe in such case you can code some additional logic to handle when the doorbell is being called, only for CGI source.

AlexxIT · 2024-05-23T04:03:32Z

RTSP backchannel has two fallbacks in case of any problems. You can increase log level for rtsp module to trace.

Check this:

go2rtc/pkg/rtsp/client.go

Lines 241 to 249 in 686fb37

    
           // some Dahua/Amcrest cameras fail here because two simultaneous 
        
           // backchannel connections 
        
           if c.Backchannel { 
        
           	c.Backchannel = false 
        
           	if err = c.Reconnect(); err != nil { 
        
           		return 0, err 
        
           	} 
        
           	return c.SetupMedia(media) 
        
           }

And this:

go2rtc/internal/rtsp/rtsp.go

Lines 126 to 140 in 686fb37

    
           if err := conn.Describe(); err != nil { 
        
           	if !conn.Backchannel { 
        
           		return nil, err 
        
           	} 
        
           	log.Trace().Msgf("[rtsp] describe (backchannel=%t) err: %v", conn.Backchannel, err) 
        
           	// second try without backchannel, we need to reconnect 
        
           	conn.Backchannel = false 
        
           	if err = conn.Dial(); err != nil { 
        
           		return nil, err 
        
           	} 
        
           	if err = conn.Describe(); err != nil { 
        
           		return nil, err 
        
           	} 
        
           }

TokarevSergey · 2024-05-23T07:43:38Z

How will he fix it? So that everything works.

AlexxIT · 2024-05-23T07:50:56Z

You take the problem device and throw it in the trash.
Buy a new device that doesn't have these firmware issues.

Or you just accept your device's firmware problems.

RTSP backchannel is a new technology for camera developers. My new Dahua camera has no such problems.

TokarevSergey · 2024-05-23T08:14:29Z

I have a Hikvision DS-KV6113-WPE1 calling panel. Are you suggesting we throw it away?

felipecrs · 2024-05-23T12:03:10Z

RTSP backchannel has two fallbacks in case of any problems. You can increase log level for rtsp module to trace.

Thank you, I will collect this information to continue the investigation.

In fact I am causing some of this confusion because I was mentioning several issues after working around the first.

I will focus on the first problem I mentioned, which is the issue description. This one does not appear to be caused by the device itself.

Therefore, @AlexxIT, I think it may be possible to replicate the issue using your current Dahua camera. But let me do things on my side first.

Also, @TokarevSergey, AFAIK Hikvision works fine using ISAPI. There was even a fix to ensure connections are closed recently in go2rtc. Regardless, let's try to keep this topic straight to a single issue. If you have something not working as expected with Hikvision, I suggest you open a different issue.

TokarevSergey · 2024-05-23T12:08:37Z

Обратный канал RTSP имеет два резерва на случай каких-либо проблем. Вы можете повысить уровень журнала для отслеживания модуля rtsp.

Спасибо, я соберу эту информацию, чтобы продолжить расследование.

На самом деле я вношу некоторую путаницу, потому что упомянул несколько проблем после работы над первой.

Я сосредоточусь на первой проблеме, которую я упомянул, а именно на ее описании. Похоже, что это не вызвано самим устройством.

Поэтому,@AlexxIT, Я думаю, что можно воспроизвести проблему, используя вашу текущую камеру Dahua. Но позвольте мне сначала сделать что-то со своей стороны.

Также,@TokarevSergey, AFAIK Hikvision отлично работает с использованием ISAPI. Недавно было даже исправлено, гарантирующее закрытие соединений в go2rtc. В любом случае, давайте попробуем свести эту тему к одному вопросу. Если у вас что-то не работает с Hikvision, я предлагаю вам открыть другую проблему.

Thanks, I've already opened a bunch of topics. Perhaps I lack the experience to explain all this correctly.

felipecrs · 2024-05-23T13:08:32Z

@AlexxIT I tried reproducing the problem with RTSP debug, but unfortunately there's nothing special going on there:

api:
  origin: '*'
log:
  rtsp: debug
streams:
  vto:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=1
    # copy PCMA track from vto
    - rtsp://127.0.0.1:8554/vto?audio=pcma
webrtc:
  candidates:
  - 192.168.1.10:8555
  - stun:8555

5/23/2024, 9:56:26 AM | info | go2rtc version=1.9.2 platform=linux/amd64 revision=b2399f3
-- | -- | --
5/23/2024, 9:56:26 AM | info | config path=/dev/shm/go2rtc.yaml
5/23/2024, 9:56:26 AM | info | [rtsp] listen addr=:8554
5/23/2024, 9:56:26 AM | info | [webrtc] listen addr=:8555/tcp
5/23/2024, 9:56:26 AM | info | [api] listen addr=:1984
5/23/2024, 9:56:30 AM | debug | [rtsp] new consumer stream=vto
5/23/2024, 9:56:58 AM | debug | [rtsp] new consumer stream=vto_hd
5/23/2024, 9:56:58 AM | debug | [rtsp] disconnect stream=vto_hd

This specific issue (not the one about ringing the doorbell) has no relationship with the device itself.

Basically, I have vto_hd depending on vto (the 2-way audio one). And if the stream is initiated on vto_hd first, it will try getting the audio track from vto without "discovering" backchannel and keep the connection to vto open.

Then, if I try to get 2-way audio with vto, it does not work. The sendonly track is not going to be "discovered" by go2rtc anymore if a previous connection without it was already open.

Just to highlight, if I try to do 2-way audio directly with vto without first opening vto_hd, everything works fine.

Finally, I found a workaround to this problem which is to have a third stream just for 2-way audio and not consume it from vto_hd:

api:
  origin: '*'
log:
  rtsp: debug
streams:
  vto:
    # H264, PCMA/8000
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=0
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=1
    # copy PCMA track from vto
    - rtsp://127.0.0.1:8554/vto?audio=pcma
  vto_2_way_audio:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
webrtc:
  candidates:
  - 192.168.1.10:8555
  - stun:8555

At least I am unblocked now. It is a bit inconvenient because then the camera.vto entity created by Frigate will not have 2-way audio and I need to tinker with stream_url on the WebRTC card or Frigat Card. Also, I suppose with Frigate 0.14 receiving native support for 2-way audio this will be even more inconvenient (not sure, I did not try).

AlexxIT · 2024-05-23T16:37:55Z

trace. Not debug

felipecrs · 2024-05-23T17:07:19Z

Ops. Here is the trace:

logs.txt

felipecrs · 2024-05-23T17:09:04Z

And here is the trace if I just open vto without opening vto_hd first:

logs.txt

Which works as expected. For comparison.

AlexxIT · 2024-05-24T10:19:17Z

Well. vto_hd should have #backchannel=0. Because this request is made first and it takes over the microphone.

1

5/23/2024, 2:05:24 PM	trace	[rtsp] client request:
DESCRIBE rtsp://192.168.1.40/cam/realmonitor?channel=1&subtype=1 RTSP/1.0
Require: www.onvif.org/ver20/backchannel

5/23/2024, 2:05:24 PM	trace	[rtsp] client response:
RTSP/1.0 200 OK

2

5/23/2024, 2:05:24 PM	trace	[rtsp] client request:
DESCRIBE rtsp://192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif RTSP/1.0
Require: www.onvif.org/ver20/backchannel

5/23/2024, 2:05:25 PM	trace	[rtsp] client response:
RTSP/1.0 404 Not Found

5/23/2024, 2:05:25 PM	trace	[rtsp] describe (backchannel=true) err: wrong response on DESCRIBE

felipecrs · 2024-05-24T12:47:56Z

It worked! And it makes total sense. So here's the working config:

streams:
  vto:
    # H264, PCMA/8000, 2-way audio
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    # copy AAC track from vto_hd
    - rtsp://127.0.0.1:8554/vto_hd?audio=aac
  vto_hd:
    # H264, AAC/16000
    - rtsp://admin:pass@192.168.1.40/cam/realmonitor?channel=1&subtype=1#backchannel=0
    # copy PCMA track from vto
    - rtsp://127.0.0.1:8554/vto?audio=pcma

Thanks a lot for the patience.

felipecrs mentioned this issue May 22, 2024

Dahua VTO support #49

Open

AlexxIT added bug Something isn't working dahua Dahua cameras, VTO, SIP labels May 23, 2024

felipecrs mentioned this issue May 23, 2024

Dahua VTO breaks the backchannel connection when someone rings it #1134

Closed

felipecrs closed this as not planned Won't fix, can't repro, duplicate, stale May 24, 2024

felipecrs mentioned this issue May 24, 2024

Add note about requesting multiple backchannel on Dahua Doorbell #1137

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When go2rtc self-requests a source capable of backchannel, it breaks backchannel #1133

When go2rtc self-requests a source capable of backchannel, it breaks backchannel #1133

felipecrs commented May 22, 2024

felipecrs commented May 22, 2024 •

edited

Loading

TokarevSergey commented May 22, 2024

felipecrs commented May 22, 2024

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 22, 2024

TokarevSergey commented May 22, 2024

felipecrs commented May 22, 2024 •

edited

Loading

AlexxIT commented May 23, 2024

TokarevSergey commented May 23, 2024

AlexxIT commented May 23, 2024

TokarevSergey commented May 23, 2024

felipecrs commented May 23, 2024

TokarevSergey commented May 23, 2024

felipecrs commented May 23, 2024

AlexxIT commented May 23, 2024

felipecrs commented May 23, 2024

felipecrs commented May 23, 2024 •

edited

Loading

AlexxIT commented May 24, 2024

felipecrs commented May 24, 2024 •

edited

Loading

When go2rtc self-requests a source capable of backchannel, it breaks backchannel #1133

When go2rtc self-requests a source capable of backchannel, it breaks backchannel #1133

Comments

felipecrs commented May 22, 2024

felipecrs commented May 22, 2024 • edited Loading

TokarevSergey commented May 22, 2024

felipecrs commented May 22, 2024

felipecrs commented May 22, 2024 • edited Loading

felipecrs commented May 22, 2024 • edited Loading

felipecrs commented May 22, 2024

TokarevSergey commented May 22, 2024

felipecrs commented May 22, 2024 • edited Loading

AlexxIT commented May 23, 2024

TokarevSergey commented May 23, 2024

AlexxIT commented May 23, 2024

TokarevSergey commented May 23, 2024

felipecrs commented May 23, 2024

TokarevSergey commented May 23, 2024

felipecrs commented May 23, 2024

AlexxIT commented May 23, 2024

felipecrs commented May 23, 2024

felipecrs commented May 23, 2024 • edited Loading

AlexxIT commented May 24, 2024

felipecrs commented May 24, 2024 • edited Loading

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 22, 2024 •

edited

Loading

felipecrs commented May 23, 2024 •

edited

Loading

felipecrs commented May 24, 2024 •

edited

Loading