Corrupted frames since v0.19.2

[jcharun]

Which version are you using?

v0.19.2

Which operating system are you using?

• Linux amd64 standard
• Linux amd64 Docker
• Linux arm64 standard
• Linux arm64 Docker
• Linux arm7 standard
• Linux arm7 Docker
• Linux arm6 standard
• Linux arm6 Docker
• Windows amd64 standard
• Windows amd64 Docker (WSL backend)
• macOS amd64 standard
• macOS amd64 Docker
• Other (please describe)

Describe the issue

Since v0.19.2, I have many artifacts and stutters when streaming with RTSP/UDP, previous version works fine.
I tried to increase readBufferCount without success.

Describe how to replicate the issue

1. start the server
2. publish with gst-launch-1.0 videotestsrc pattern=snow ! x264enc ! rtspclientsink location=<uri> or a file source
3. read with gst-launch-1.0 playbin uri=<uri>

Did you attach the server logs?

2022/07/20 14:06:54 INF rtsp-simple-server v0.19.2
2022/07/20 14:06:54 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2022/07/20 14:07:15 INF [RTSP] [conn A.B.C.D:42549] opened
2022/07/20 14:07:16 INF [RTSP] [session 926554632] created by A.B.C.D:42549
2022/07/20 14:07:17 INF [RTSP] [session 926554632] is publishing to path '1234_abcdef', 1 track with UDP
2022/07/20 14:07:17 INF [RTSP] [conn A.B.C.D:12230] opened
2022/07/20 14:07:17 INF [RTSP] [session 581669966] created by A.B.C.D:12230
2022/07/20 14:07:18 INF [RTSP] [session 581669966] is reading from path '1234_abcdef', 1 track with UDP
2022/07/20 14:07:32 INF [RTSP] [session 581669966] destroyed (teared down by A.B.C.D:12230)
2022/07/20 14:07:32 INF [RTSP] [conn A.B.C.D:12230] closed (EOF)
2022/07/20 14:07:32 INF [RTSP] [conn A.B.C.D:42549] closed (EOF)
2022/07/20 14:07:42 INF [RTSP] [session 926554632] destroyed (no UDP packets received in a while)

Did you attach a network dump?

no

[tmatth]

We've also observed the stuttering, in case its useful here is the workaround we are using in gortsplib:

From 4cf50d810879d84222e08b596882c2071b641ec3 Mon Sep 17 00:00:00 2001
From: Lionel Nicolas <lionel.nicolas@adenops.com>
Date: Fri, 1 Jul 2022 14:14:48 -0400
Subject: [PATCH] rtpcleaner: bypass h264

This is causing rtsp-simple-server to perform terribly when faced with upstream packet loss.
---
 pkg/rtpcleaner/cleaner.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pkg/rtpcleaner/cleaner.go b/pkg/rtpcleaner/cleaner.go
index 6a658b6..f3687c2 100644
--- a/pkg/rtpcleaner/cleaner.go
+++ b/pkg/rtpcleaner/cleaner.go
@@ -42,7 +42,7 @@ func NewCleaner(isH264 bool, isTCP bool) *Cleaner {
 		isTCP:  isTCP,
 	}
 
-	if isH264 {
+	if false {
 		p.h264Decoder = &rtph264.Decoder{}
 		p.h264Decoder.Init()
 	}

bluenviron/gortsplib@4cf50d8

bluenviron/gortsplib@main...tmatth:gortsplib:h264-workarounds

[aler9]

Hello all,

After an assessment i've realized that since v0.19.2, RTP packets are discarded if the H264 decoding fails, and this causes troubles in case of packets losses. RTP packets should be routed even if the server is not able to decode them.
So this will be changed ASAP, thanks @tmatth for pointing to the exact reason.

Anyway, v0.19.3 will include the new UDP packet reordering mechanism (https://github.com/aler9/gortsplib/blob/main/pkg/rtpreorderer/reorderer.go), that reorders packets before they're processed. This will surely decrease decoding errors.

[tmatth]

Hello all,

After an assessment i've realized that since v0.19.2, RTP packets are discarded if the H264 decoding fails, and this causes troubles in case of packets losses. RTP packets should be routed even if the server is not able to decode them. So this will be changed ASAP, thanks @tmatth for pointing to the exact reason.

Credit to @lionelnicolas for debugging and finding the workaround, we will follow up with testing with v0.19.3 as we would like to drop these workarounds eventually.

[jcharun]

Anyway, v0.19.3 will include the new UDP packet reordering mechanism

Will it add extra latencies? If so, can we disable it in config please?

[aler9]

Will it add extra latencies? If so, can we disable it in config please?

UDP reordering is present in both GStreamer and FFmpeg. Their implementation increases latency, the RTSPSS one doesn't increase latency. If the server receives packets with these sequence numbers:

1
2
4
3
5

They will be processed in this order:

1
2
3
4
5

This feature may increase the number of dropped packets. For instance, let's suppose the server receives packets with these sequence numbers:

1
2
4
5

Packets with sequence number 4 and 5 will be put on hold until packet with sequence number 3 is received. There's a limit on the number of buffered packets, that is 64.

A way to solve this issue consists into implementing negative acknowledges (NACKs), i.e. telling the counterpart to re-send packet 3, but the problem is that NACKs are rarely supported by cameras.

(sorry, i wrote a thing that is not true, if the number of buffered packets exceeds 64, the entire buffer is released and sent to readers).

[aler9]

added in v0.19.3

[github-actions]

This issue is being locked automatically because it has been closed for more than 6 months.
Please open a new issue in case you encounter a similar problem.