Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Coredump when 1.3.4 client send srtpacket to 1.4.1 server #1182

Closed
kyolong opened this issue Mar 15, 2020 · 6 comments · Fixed by #1315
Closed

[BUG]Coredump when 1.3.4 client send srtpacket to 1.4.1 server #1182

kyolong opened this issue Mar 15, 2020 · 6 comments · Fixed by #1315
Assignees
Labels
[core] Area: Changes in SRT library core Type: Bug Indicates an unexpected problem or unintended behavior
Milestone

Comments

@kyolong
Copy link
Contributor

kyolong commented Mar 15, 2020

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:
FFmpeg 4.2.2 with libsrt 1.3.4
my SrtServer, ues libsrt 1.4.1

1.about 20 connections between my server and ffmpeg, when i kill -9 ffmpeg, srt_epoll triggle EPOLL_ERROR, than i remove srt fd from epoll, this time coredump. i can not get any coredump file.
2. sometimes, when i close some srt connections, ffmpeg also coredump, forllow is detail info

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fac2a2e9238 in CUDT::packData(CPacket&, unsigned long&) ()
from /data/jzj/bvc_network_srt/libsrt.so.1
[Current thread is 1 (Thread 0x7fac28445700 (LWP 8783))]
(gdb) bt
#0 0x00007fac2a2e9238 in CUDT::packData(CPacket&, unsigned long&) ()
from /data/jzj/bvc_network_srt/libsrt.so.1
#1 0x00007fac2a3054a3 in CSndUList::pop(sockaddr*&, CPacket&) ()
from /data/jzj/bvc_network_srt/libsrt.so.1
#2 0x00007fac2a3055aa in CSndQueue::worker(void*) () from /data/jzj/bvc_network_srt/libsrt.so.1
#3 0x00007fac2b09b6ba in start_thread (arg=0x7fac28445700) at pthread_create.c:333
#4 0x00007fac2add141d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)

[Sun Mar 15 04:56:51 2020] SRT:SndQ:worker[24005]: segfault at 188 ip 000000000077c0f4 sp 00007f3cd7ffe920 error 4 in bvc-srt-server[400000+87e000]
[Sun Mar 15 05:02:47 2020] SRT:SndQ:worker[31981]: segfault at 188 ip 000000000077c0f4 sp 00007f3cd7ffe920 error 4 in bvc-srt-server[400000+87e000]
[Sun Mar 15 05:53:05 2020] SRT:SndQ:worker[3589]: segfault at 188 ip 000000000077c0f4 sp 00007f3cb27fb920 error 4 in bvc-srt-server[400000+87e000]
[Sun Mar 15 07:15:10 2020] SRT:SndQ:worker[18327]: segfault at 188 ip 00007feb15346238 sp 00007feb134a1b70 error 4 in libsrt.so.1.3.4[7feb15301000+83000]
[Sun Mar 15 10:23:09 2020] SRT:SndQ:worker[15297]: segfault at 188 ip 00007fc84879e238 sp 00007fc8468f9b70 error 4 in libsrt.so.1.3.4[7fc848759000+83000]
[Sun Mar 15 11:15:15 2020] SRT:SndQ:worker[22244]: segfault at 188 ip 00007fa2fc812238 sp 00007fa2fa96db70 error 4 in libsrt.so.1.3.4[7fa2fc7cd000+83000]
[Sun Mar 15 12:30:41 2020] SRT:SndQ:worker[28992]: segfault at 188 ip 00007fc10cf14238 sp 00007fc10b06fb70 error 4 in libsrt.so.1.3.4[7fc10cecf000+83000]
[Sun Mar 15 13:01:41 2020] SRT:SndQ:worker[6687]: segfault at 188 ip 000000000077c0f4 sp 00007f3cd7ffe920 error 4 in bvc-srt-server[400000+87e000]
[Sun Mar 15 13:10:44 2020] SRT:SndQ:worker[19137]: segfault at 188 ip 000000000077c0f4 sp 00007f3ce4ff8920 error 4 in bvc-srt-server[400000+87e000]
[Sun Mar 15 13:12:48 2020] SRT:SndQ:worker[22193]: segfault at 188 ip 000000000077c0f4 sp 00007f3cb2d0b920 error 4 in bvc-srt-server[400000+87e000]

@kyolong kyolong added the Type: Bug Indicates an unexpected problem or unintended behavior label Mar 15, 2020
@ethouris
Copy link
Collaborator

Do you use all 20 connections at a time to perform transmission? Could you give me a more detailed description of how you run this ffmpeg and more-less what you are doing on your SrtServer (who the caller is, what are socket options etc.)? I'll be trying to reproduce this, but I need this extra information.

If you could use a debug version of SRT (also the release-debug version, available with --enable-debug=2) in your SrtServer and repeat this, that might be even more useful.

@kyolong
Copy link
Contributor Author

kyolong commented Mar 17, 2020

Srt server is listener, ffmpeg is caller.
srt config :
live mode
set connect_timeout -> 100000 ms

set payloadsize -> 1456

set latency -> 200

set drop -> 0

set tsbpd -> 0

set maxbw -> 2097152

Today i make a test,instead of ffmpeg.
SRT Server just recv ,not do other things.
client connect to server with 200 connections.

Launch 1 client connect to srt server,the transmissions is very OK, about 400kbps per connection. About 10s later, client call srt_cloese() for every connections.
Most of the time . client is OK, srt server is also OK.
but , sometimes client will coredump at CUDT::packData(CPacket&, unsigned long&) ()

I test about 100 times, can reproduce once.

sample :
int srt_socket_list[100];
for(i=0;i<200;i++)
{
     srt_socket_list[i] = srt_create_socket();
      srt_connect(srt_socket_list[i] =, xxxx, xxxxxx);
}
for(j=1;j<200;j++)
{
     for(i=0;i<100;i++)
     {
         srt_send(srt_socket_list[i],buf,size);
     }
}
for(i=0;i<200;i++)
{
      srt_close(srt_socket_list[i]);
}
`

@maxsharabayko maxsharabayko added the [core] Area: Changes in SRT library core label Mar 18, 2020
@maxsharabayko maxsharabayko added this to the v1.5.0 milestone Mar 18, 2020
@kyolong
Copy link
Contributor Author

kyolong commented Mar 18, 2020

0x00000000007bbe74 in CCryptoControl::getSndCryptoFlags (this=0x0)
at /data/app/bvc-srt-server/bvc_network_srt/srtcore/crypto.h:226
#1 0x00000000007b1872 in CUDT::packData (this=0x7fad8ce9e490, packet=...,
ts_tk=@0x7fad5d7f9780: 1584530038433899)
at /data/app/bvc-srt-server/bvc_network_srt/srtcore/core.cpp:8021
#2 0x00000000007e225a in CSndUList::pop (this=0x7fad8d045310,
addr=@0x7fad5d7f99e0: 0x7fa9e22e06e0, pkt=...)
at /data/app/bvc-srt-server/bvc_network_srt/srtcore/queue.cpp:328
#3 0x00000000007e2e6d in CSndQueue::worker (param=0x7fad8cf71ea0)
at /data/app/bvc-srt-server/bvc_network_srt/srtcore/queue.cpp:600
#4 0x00007fada21b16ba in start_thread (arg=0x7fad5d7fa700) at pthread_create.c:333
#5 0x00007fada144241d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

@kyolong
Copy link
Contributor Author

kyolong commented Mar 20, 2020

fix it in new PR #1193 @ @maxsharabayko @ethouris

@maxsharabayko
Copy link
Collaborator

Hi @kyolong
This issue was fixed by PR #1315.
Could you please confirm you don't get this crash with the latest master?

@ethouris
Copy link
Collaborator

ethouris commented Jun 4, 2020

@kyolong I added a UT #1323, which should more-less reproduce the steps you provided to reproduce the problem.

In my environment in the latest version I couldn't reproduce the problem, though. Have idea for an improvement?

@mbakholdina mbakholdina modified the milestones: v1.5.0 - Sprint 16, v1.4.2 Oct 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[core] Area: Changes in SRT library core Type: Bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants