Edge: FFprobe may get stuck when pulling stream from edge. #2594
Comments
This comment has been minimized.
This comment has been minimized.
|
Test code for Go package main
import (
"bufio"
"bytes"
"errors"
"io"
"os/exec"
"syscall"
)
func processCmdPipeOut(pstdout *[]byte, pout io.ReadCloser, endpipe chan bool) {
r := bufio.NewReader(pout)
for {
//data, err := r.ReadBytes('\r')
data := []byte{}
var c byte
var e error
for {
c, e = r.ReadByte()
if e != nil {
break
}
data = append(data, c)
if c == '\r' || c == '\n' {
break
}
}
if len(data) > 0 {
print(string(data))
*pstdout = append(*pstdout, data...)
}
if e != nil {
break
}
}
endpipe <- true
}
func RunCmdWithCallback(cmdline string) ([]byte, error) {
var stdout, stderr []byte
var err error = nil
cmd := exec.Command("/bin/sh", "-c", cmdline)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
pout, err0 := cmd.StdoutPipe()
perr, err1 := cmd.StderrPipe()
if err0 != nil || err1 != nil {
return nil, errors.New("pipe error")
}
chnum := 2
endpipe := make(chan bool, chnum)
defer close(endpipe)
cmd.Start()
// read data from StdoutPipe
go processCmdPipeOut(&stdout, pout, endpipe)
// read data from StderrPipe
go processCmdPipeOut(&stderr, perr, endpipe)
err = cmd.Wait()
for i := 0; i < chnum; i++ {
<-endpipe
}
return append(stdout, stderr...), err
}
func RunCmd2(cmdline string) ([]byte, []byte, error) {
cmd := exec.Command("/bin/sh", "-c", cmdline)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
wo := bytes.NewBuffer(nil)
we := bytes.NewBuffer(nil)
cmd.Stdout = wo
cmd.Stderr = we
err := cmd.Run()
stdout := wo.Bytes()
stderr := we.Bytes()
return stdout, stderr, err
}
func main() {
ffprobeCmd := "ffprobe -hide_banner -rw_timeout 10000000 -of json -show_entries 'format:stream' -i 'rtmp://192.168.1.143:1935/test/lmt?vhost=test-play.live.com'"
sout, serr, _ := RunCmd2(ffprobeCmd)
println(string(sout))
println(string(serr))
ffmpegCmd := "ffmpeg -hide_banner -rw_timeout 10000000 -i 'rtmp://192.168.1.143:1935/test/lmt?vhost=test-play.live.com' -c:a copy -c:v libx264 -g 60 -preset veryfast -threads 8 -f flv -y 'rtmp://192.168.1.143:1935/test/lmt_trans?vhost=test-push.live.com'"
RunCmdWithCallback(ffmpegCmd)
}
|
|
After taking a look, I didn't understand why the publish timeout occurred. Can you find the minimal reproducible path? For example, does this issue exist with a single SRS?
|
I haven't found a better way to reproduce it yet. I suspect it may be a problem with the code in the Go coroutine that retrieves the StdoutPipe data, causing the log call in ffmpeg to block. I fixed it by modifying the Go code. However, it's also strange that this issue only occurs with RTMP, and there is no such phenomenon when using HTTP-FLV to pull the stream.
|
|
Temporary solution found. It is the same phenomenon as this issue. In the GO code, after ffprobe, ffmpeg is immediately called. When ffprobe's connection is disconnected, the edge has not yet completed the logic of on_all_client_stop, causing ffmpeg's connection to come up. The SRS edge status is still "init", which will cause ffmpeg to hang.
|
|
Please do not comment randomly in the Issue, and do not go off-topic. Each Issue should only address one problem, otherwise it will be deleted directly. Before submitting an Issue, be sure to read FAQ #2716.
|
First of all, kudos to you. The cause of the issue has been identified. It seems that it would be better to return an error in this situation. When in an intermediate state, returning an error to the client can prevent it from getting stuck.
|
winlinvip
changed the title
winlinvip
changed the title
This comment has been minimized.
This comment has been minimized.
winlinvip
changed the title
winlinvip
changed the title
winlinvip
changed the title
|
To fix the incorrect state of edge, we should refine the state machine of source, changing between publishing and unpublishing like DB transaction. |
Description'
Please ensure that the markdown structure is maintained.
The first time using exec.Command in Go to execute FFprobe, and then executing FFmpeg, while using goroutines to read FFmpeg's StdoutPipe and StderrPipe, there is a probability of causing the streaming to freeze. If executing ffmpeg -i rtmp address in the terminal, the streaming freeze issue is automatically resolved. This phenomenon occurs when pulling streams from the edge vhost, but there is no such issue when pushing streams to the vhost.
5.0.11Replay
1. Push the stream in the terminal using ffmpeg -re -stream_loop -1 -i ~/Documents/live_test.mp4 -c copy -f flv rtmp://test-push.live.com/test/lmt
2. Execute the following command using Go's exec.Command
3. Then, execute the following command using exec.Command and start a goroutine to read data from StdoutPipe and StderrPipe
4. The streaming gets stuck, and the SRS log shows "publish timeout", which can be reproduced with a certain probability. If it cannot be reproduced, you need to close the streaming and go process, and then start again from step 1.
Expected Behavior (Expect)
Able to stream normally.
TRANS_BY_GPT3The text was updated successfully, but these errors were encountered: