OBS Detect - Object Detection and Masking Filter

A plugin for OBS Studio that allows you to detect many types of objects in any source, track them and apply masking.

If you like this work, which is given to you completely free of charge, please consider supporting it by sponsoring us on GitHub:

• https://github.com/sponsors/royshil
• https://github.com/sponsors/umireon

This work uses the great contributions from EdgeYOLO-ROS and PINTO-Model-Zoo. The Hungarian algorithm is taken from https://github.com/Gluttton/munkres-cpp under the GPLv2 license.

Usage

 

• Add the "Detect" filter to any source with an image (Media, Browser, VLC, Image, etc.)
• Enable "Masking" or "Tracking"

Use Detect to track your pet, or blur out people in your video!

More information and usage tutorials to follow soon.

Features

Current features:

• Detect over 80 categories of objects, using an efficient model (EdgeYOLO)
• 3 Model sizes: Small, Medium and Large
• Face detection model, fast and efficient (YuNet)
• Load custom ONNX detection models from disk
• Filter by: Minimal Detection confidence, Object category (e.g. only "Person"), Object Minimal Size
• Masking: Blur, Pixelate, Solid color, Transparent, output binary mask (combine with other plugins!)
• Tracking: Single object / Biggest / Oldest / All objects, Zoom factor, smooth transition
• SORT algorithm for tracking smoothness and continuity
• Save detections to file in real-time, for integrations e.g. with Streamer.bot

Roadmap features:

• Precise object mask, beyond bounding box
• Multiple object category selection (e.g. Dog + Cat + Duck)
• Make available detection information for other plugins through settings

Train and use a custom detection model

Follow the instructions in docs/train_model.md to train and use your own custom model.

Building

The plugin was built and tested on Mac OSX (Intel & Apple silicon), Windows and Linux.

Start by cloning this repo to a directory of your choice.

Mac OSX

Using the CI pipeline scripts, locally you would just call the zsh script. By default this builds a universal binary for both Intel and Apple Silicon. To build for a specific architecture please see .github/scripts/.build.zsh for the -arch options.

$ ./.github/scripts/build-macos -c Release

Install

The above script should succeed and the plugin files (e.g. obs-ocr.plugin) will reside in the ./release/Release folder off of the root. Copy the .plugin file to the OBS directory e.g. ~/Library/Application Support/obs-studio/plugins.

To get .pkg installer file, run for example

$ ./.github/scripts/package-macos -c Release

(Note that maybe the outputs will be in the Release folder and not the install folder like pakage-macos expects, so you will need to rename the folder from build_x86_64/Release to build_x86_64/install)

Linux (Ubuntu)

Use the CI scripts again

$ ./.github/scripts/build-linux.sh

Copy the results to the standard OBS folders on Ubuntu

$ sudo cp -R release/RelWithDebInfo/lib/* /usr/lib/x86_64-linux-gnu/
$ sudo cp -R release/RelWithDebInfo/share/* /usr/share/

Note: The official OBS plugins guide recommends adding plugins to the ~/.config/obs-studio/plugins folder.

Windows

Use the CI scripts again, for example:

> .github/scripts/Build-Windows.ps1 -Target x64

The build should exist in the ./release folder off the root. You can manually install the files in the OBS directory.