This project makes use of Yolo v4 and the Darknet framework. Nowadays there more efficient and precise ways to perform object inference with GStreamer, for instance by using Nvidia Deepstream with Yolo v5, YOLO-R, etc. I suggest you to take a look at these.
gst-darknet is a GStreamer plugin that allows to use Darknet (neural network framework) inside GStreamer, to perform object detection against video files or real-time streams. For instance, the video above was generated with the following command:
gst-launch-1.0 \
filesrc location=test.mp4 ! decodebin ! videoconvert \
! darknetinfer config=yolov4.cfg weights=yolov4.weights \
! darknetrender labels=coco.names \
! videoconvert \
! xvimagesink sync=1
The plugin provides these elements:
darknetinfer
, that runs Darknet against one or multiple input videosdarknetprint
, that prints the detected objects to stdoutdarknetrender
, that draws the detected objects on the input video
-
Install build dependencies:
sudo apt update && sudo apt install -y \ git \ make \ pkg-config \ g++ \ libgstreamer-plugins-base1.0-dev \ libcairo2-dev
-
Clone the repository, enter into the folder:
git clone https://github.com/aler9/gst-darknet \ && cd gst-darknet
-
Compile and install:
make -j$(nproc) \ && sudo make install
-
Download config, weights, class labels and a sample video:
wget https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg \ && wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights \ && wget https://raw.githubusercontent.com/AlexeyAB/darknet/master/data/coco.names \ && wget https://raw.githubusercontent.com/aler9/gst-darknet/master/test/test.mp4
-
Launch the pipeline:
gst-launch-1.0 \ filesrc location=test.mp4 ! decodebin ! videoconvert \ ! darknetinfer config=yolov4.cfg weights=yolov4.weights \ ! darknetrender labels=coco.names \ ! videoconvert \ ! xvimagesink sync=1
If the platform is a Nvidia Jetson:
gst-launch-1.0 \ filesrc location=test.mp4 ! decodebin ! nvvideoconvert ! videoconvert \ ! darknetinfer config=yolov4.cfg weights=yolov4.weights \ ! darknetrender labels=coco.names \ ! videoconvert \ ! xvimagesink sync=1
Launch the pipeline:
gst-launch-1.0 \
rtspsrc location=rtsp://myurl:554/mypath ! decodebin ! videoconvert \
! darknetinfer config=yolov4.cfg weights=yolov4.weights \
! darknetrender labels=coco.names \
! videoconvert \
! xvimagesink sync=1
Launch the pipeline:
gst-launch-1.0 \
filesrc location=test.mp4 ! decodebin ! videoconvert \
! darknetinfer config=yolov4.cfg weights=yolov4.weights \
! darknetprint labels=coco.names \
! fakesink
Launch the pipeline:
gst-launch-1.0 \
filesrc location=test.mp4 ! decodebin ! videoconvert \
! darknetinfer config=yolov4.cfg weights=yolov4.weights \
! darknetrender labels=coco.names \
! videoconvert \
! x264enc \
! mp4mux \
filesink location=output.mp4
Launch the pipeline:
gst-launch-1.0 \
darknetinfer name=d config=yolov4.cfg weights=yolov4.weights \
multifilesrc location=dog.jpg caps="image/jpeg,framerate=20/1" ! jpegdec ! videoconvert \
! d.sink_0 d.src_0 ! darknetrender labels=coco.names ! videoconvert ! xvimagesink sync=1 \
multifilesrc location=giraffe.jpg caps="image/jpeg,framerate=20/1" ! jpegdec ! videoconvert \
! d.sink_1 d.src_1 ! darknetrender labels=coco.names ! videoconvert ! xvimagesink sync=1
darknetinfer
properties:
gpu-id
: GPU to use for inference (default is 0)config
: path to a Darknet config fileweights
: path to a Darknet weights fileprobability-threshold
: probability threshold of detected objects (default is 0.7)nms-threshold
: NMS threshold of detected objects (default is 0.45)print-fps
: periodically print FPS to stdout (default is TRUE)print-fps-period
: Period of FPS printing in seconds (default is 5)
darknetprint
properties:
labels
: path to a label file
darknetrender
properties:
labels
: path to a label filebox-color
: color of the boxes in HTML format (default is 00FFFF)text-color
: color of the text in HTML format (default is 000000)
One of the ways to export detections consists in launching GStreamer through a C program that makes use of the GStreamer API. The detections are then available in a C struct (GstDarknetMetaDetection
) and can be exported in any desired way.
-
Copy examples/export.c in an empty folder, edit to suit needs.
-
Compile (replace
/path-to-gst-darknet
):gcc -Ofast -Werror -Wall -Wextra -Wno-unused-parameter \ -I/path-to-gst-darknet \ export.c -o export \ $(pkg-config --cflags --libs gstreamer-app-1.0)
-
Launch:
./export
- Darknet (AlexeyAB) https://github.com/AlexeyAB/darknet
- Darknet (pjreddie) https://github.com/pjreddie/darknet