Video+Audio in same packet??

[alatteri]

Hi UG team,

I'm wondering if there is a mode to have the video+audio packets encapsulated together instead of as separate "streams"? Since I encapsulate the output of UG into SRT, I currently have 2 SRT streams, video + audio. Sometimes this can lead to sync issues. It would be great to have video+audio together to help simplify this setup and avoid sync issues. I believe MPEGTS does this, but AFAIK, UG uses its own format. I'm doing H.265+AAC.

Thanks.

[MartinPulec]

Hi @alatteri,
exactly as you say, currently no directly with UG. You can try either emitting NDI (with an NDI display) or there is a branch with experimental "own" SRT transmit (it is /at least currently/ incomatible with 3rd party SW). That bundles audio with the video, but it is yet in an alpha stage. MPEG-TS would be a great idea, I agree, but it is mostly a long way run and thus a matter of priorities so I cannot promise that we will do that.

[alatteri]

Hi @MartinPulec

Thanks for the info. I use reflector to send to multiple recipients at the same time, so I assume SRT would not work in that case.
I'll keep things the way they are.

Alan

[mikekoetter]

Martin have you ever considered writing out to a NUT format FIFO pipe, basically the inverse of the example you give for how to pipe into UltraGrid?

[mikekoetter]

Shortly after I posted this I found the unix_socket display, is that essentially doing that except over unix sockets?
I just tried connecting it up to an FFMPEG input, but I'm getting an "Invalid data found when processing input" error

[mpiatka]

The unix_socket display does output video frames over unix sockets, but it's primary purpose was the video preview in the GUI. It uses a custom header with uncompressed RGB data. So it's not something ffmpeg can read and is probably too limited for your use case. But just in case you want to try, you can find some code in the tools directory (particularly tools/ipc_frame_unix.h).

[MartinPulec]

@mikekoetter please create a separate discussion thread or an issue on this topic if you want to continue with the discussion or create a feature request. This thread is not related and I think we should keep the things organized.