Open Source Reproduction: How OBS Supports Live Multi-Person Connection, Online Press Conference with Connection Capability

[winlinvip]

How does OBS support live multi-person connection and BGM? For example, how is the online press conference solution of SRS 4.0 with multi-person connection, multi-platform live streaming, and multi-scene switching implemented?

Without further ado, let's get straight to the point. I recorded a 3-minute video tutorial Bilibili: OBS Connection and Multi-Person Connection, which is easy to use and can be explained in 3 minutes.

Timmerse needs to use a special version, you can scan the QR code in the video to join the group and get it.

Goal

A picture is worth a thousand words, and the goal is to achieve the capabilities shown in the image below, using OBS for live streaming:

Note: Related other links: Zhihu: How does OBS achieve multi-screen connection?, Zhihu: How does obs live streaming achieve multi-person remote live streaming?, Zhihu: What are the advantages of using OBS for video live streaming?, B Station: How does OBS live streaming connect and multi-person connect, CSDN: How does OBS live streaming connect and multi-person connect, Jianshu: How does OBS live streaming connect and multi-person connect

How

I will share the specific implementation and Demo in detail, and of course, I will arrange Q&A.

To avoid unclear images in the video or untimely updates, I will describe the general technical solution here, which will be explained in detail in the video.

Overall deployment architecture diagram, a host and a computer can complete an online press conference. Of course, some auxiliary tools are needed, including SRS, FFmpeg, video account, B station account, connection tool (Timmerse or Tencent conference or other).

Detailed architecture diagram reference SRS 2021 Online Press Conference Technical Solution

The general system structure diagram refers to Figma: OBS Connection

Why

Why is it OBS connection, not the connection of various live streaming platforms, or the connection function of conference SaaS?

First of all, there are still many OBS users, most of whom are doing live streaming, a single host doing live streaming. In entertainment live streaming, show venues, education, events, live events, e-commerce, games, etc., OBS can be used for live streaming everywhere.

The question is, why use OBS for live streaming? Is OBS's core capability live streaming?

No!

OBS's core capability is its relatively rich director capabilities. The relative here is in comparison to professional broadcast-level director consoles, OBS is definitely not good enough. The problem is that we don't need such a professional director console, right? Now live streaming is a popular capability, everyone can use live streaming, and everyone will do live streaming in the future, so does everyone need to buy a director console? ! There is no need.

What exactly does director capability mean? For example, scene switching, as shown below:

In the lower left corner are Scenes, and live events must have a preview, such as SRS live streaming every Saturday night at 20:00, so you can't wait until 20:00 to start streaming, right? It may start preparing and streaming at 19:50, and the preview will be played for about 5 to 10 minutes, which can also include promotional videos or event descriptions, etc., and the key is to have BGM. So I defined two Scenes:

• WeeklyBGM: Live preview, which will have BGM (background music), usually, I use Victor (recommended by Master Xiong). And at this time, the microphone cannot be collected, as it may not be convenient to collect, and everyone is debugging and the like.
• WeeklyLive: Official live streaming, you can see the collection of the connection window, under the temporary onlookers group QR code is a multi-person connection avatar list, this is implemented using Timmerse recommended by Xiangzong (other solutions can be used, detailed We will explain in the reproduction video).

Now it's 20:00, just click the Transition button between the two scenes, and the screen will smoothly switch over, and you can set the switching time and so on. This is one of OBS's core capabilities: multi-scene directing.

During live streaming, sometimes we need to temporarily adjust the live room settings, such as temporarily checking the guest's work, so we can discuss other topics first, while the staff create a new scene, and when it's ready, switch to the new scene. This is a feature I often use, how convenient!

Is OBS's scene switching capability the only one? There are many sources, such as video files, windows, sub-scenes, text, VLC sources, images, image sets, video devices, audio devices, etc. And filters can be added to these.

Filters are a super awesome feature. For example, one problem is that the computer's microphone will have fan noise, and at this time you can use RNN noise reduction to only retain the human voice. For example, you can do green screen keying, yes, OBS can also do green screen keying. For example, you can crop the window size and only display a certain part, such as I often capture OBS's own bottom status bar into the video stream, and you can see the real-time live streaming bitrate in the live room.

Such a good live streaming artifact, and it's open source, with a wealth of plugins, a huge number of plugins. Is it necessary to reinvent a rounder wheel? It may be necessary to reinvent one, but it's really hard to make it rounder. OBS is a very, very old open source project, with 33K stars, 8 years of time, and very active, as shown below:

Alright, let's make it clear why we need to use OBS. Why do we need to use OBS for co-broadcasting? Because sometimes live streaming requires co-broadcasting. At present, most of the live streams that can be co-broadcasted are implemented by the platforms themselves, such as video number co-broadcasting, which cannot be pushed to Bilibili afterwards. Or using Tencent Meeting for co-broadcasting can be pushed to other live streaming platforms, but it cannot use OBS's capabilities and cannot be pushed to multiple platforms. That is, the current live co-broadcasting cannot be used in OBS, and you have to do it yourself, which is why everyone is curious about how SRS does it.

However, OBS does not yet have co-broadcasting capabilities, so we came up with a solution that is relatively simple to implement OBS live co-broadcasting. The goals are as follows:

• Use OBS for live streaming, using OBS's powerful directing, rich features and filters, and familiar tools.
• There must be BGM, co-broadcasting, mixing, and intelligent noise reduction for computer fan noise, and the sound must not be messed up.
• Co-broadcasting guests should preferably have video avatars, but if the business only requires sound, it can also reduce bandwidth consumption.

It's quite difficult to have no team guarantee, low cost, and live multi-person co-broadcasting without messing up, haha.

PS: Since OBS is used for streaming, multi-platform live streaming can also be done, which is a common feature in live streaming. Of course, we also have an open-source replica to share with you, see #2676.

Air Pods

Because there is live streaming and co-broadcasting, the host must wear headphones, otherwise, various sound problems will occur.

The host's AirPods must turn off the in-ear detection (Automatic In-Ear Detection), otherwise putting the headphones back on will cause the Output to switch, resulting in the live stream possibly having no sound.

Latency

The latency of live co-broadcasting is relatively easy to measure. After using the camera to capture the stopwatch, playing each link can see the latency comparison.

Each link will add some latency, and the total latency to the audience is about 5~7 seconds. OBS collection (including co-broadcasting) latency is about 800ms. OBS streaming to the local SRS latency is about 200ms, including encoding and transmission. FFmpeg forwarding to the cloud platform and playback has about 4 seconds of latency.

In the co-broadcasting latency, it is about 800ms, including RTC latency and OBS collection latency. The RTC latency is about 200ms. OBS collection also has latency, such as capturing the RTC co-broadcasting window, which will have about 600ms of latency.

Timmerse

In fact, Timmerse can also be implemented with open-source solutions:

• The client uses WebRTC, which can be done with H5.
• The backend uses SRS, and SFU forwarding implements multi-person co-broadcasting.

Since it involves a lot of business customization, it will not be implemented for now, as it is too troublesome.

[winlinvip]

Reference: Open Source Replica: How to Co-broadcast with OBS Live, Concluded!