Architecture

Applications architecture

The motion camera consists of four parts.
All four run in a separate process, having their unique PID. We use processes instead of threads to make sure there isn't a time-consuming thread holding an image stream. The last thing you want is for the app to respond to long-ago events because of an overflow in an image buffer.

1. The first part is the FFmpeg streamer and the Nginx server. The FFmpeg wraps the h264 video stream to an RTMP friendly flv format. The Nginx with the RTMP add-on converts the video to HLS, a modern browser format. You can watch the video in any browser. The FFmpeg software only uses the GPU, a great advantage when you later need all the CPU power available for deep learning models. The Nginx server also runs very quietly in the background, consuming a max of 1% CPU.
2. With inference times up to 0.5 sec, timing is crucial. You cannot have any buffering in the chain. Every image processed needs to be with as little latency as possible. Here, the Qstreamer comes in place. It constants grab frames from the HLS stream FFmpeg and Nginx generates. Once an image is processed, a flag is set, indicating a request for a new frame. The Qstreamer will respond to the request by sending the latest frame just received.
3. MotionEvent analyses the images. Once a movement is detected it send the appropiate signals to other parts of the application.
4. The last part is the recording. If a USB memory device is connected, the recorder will use it. If not, they are stored at /home/pi/Videos. If Gdrive is enabled, the recording uploads upon completion. The original recording is deleted if this option is set. Be careful when using the overlay feature (the Raspberry Pi's read-only safety mode). All writes are in RAM. Make sure you have enough space. The program does not check this. Another FFmpeg stream takes care of the recording. This way, you also save the images 5 seconds before the movement event. In other words, you see the postman walking up the porch instead of just walking out of your yard just because of a time-lapse when the shooting starts. The latter will be the case if you use Nginx's recording facilities.