Using the RPi's Video Codecs and Camera Module

Easily leverage your RPi4/3's built-in video codecs, and optional camera module, from the command line!

Introduction

As of v1.4.0 of the image, you can use your RPi3's built-in hardware video codecs, and also capture video streams and stills from the Raspberry Pi camera module, if you have one attached, via V4L2. As of v1.5.0, this works with an RPi4 as well. And, as of v1.5.2, MMAL is also available (together with bundled apps such as raspivid and raspistill).

While the image includes two demo applications illustrating camera and codec access (Applications→Multimedia→RPi Camera Live View and Applications→Multimedia→RPi Video Player (HW Codecs)), these features can easily be exploited from the command line too. This page just collects together a few 'recipes' for common operations. Feel free to add your own!

NB, as of the time of writing, a patched version of ffmpeg is required to access V4L2 endpoints; the version supplied on the current image has had the necessary patches applied, via the v4l2m2m-fix USE flag. And, as of v1.5.2, the mmal USE flag is set too, so MMAL endpoints can also be accessed.

Prerequisites

To use the hardware codecs via v4l2 m2m or MMAL, you must have:

gpu_mem=96

or greater in /boot/config.txt (you can use the bundled Applications→Settings→RPi Config tool to set this, if desired); requires a reboot to take effect, if changed.

If you want to use the camera module, you must have:

gpu_mem=128

or greater, and:

start_x=1

set in /boot/config.txt.

Capturing Output from the Camera

To save an h264 stream from the camera to file, issue (e.g.):

demouser@pi64 ~ $ ffmpeg -f video4linux2 -input_format h264 \
  -video_size 1280x720 -framerate 30 -i /dev/video0 \
  -vcodec copy -an -f matroska test.mkv

Ctrlc to stop the grab when done.

As with all recipes here, feel free to vary the parameters as required.

Or, to grab (e.g.) 10 numbered jpeg files in sequence from the camera at 5fps, issue:

demouser@pi64 ~ $ ffmpeg -f video4linux2 -input_format mjpeg \
  -video_size 1280x720 -framerate 5 -i /dev/video0 \
  -vcodec copy -an -vframes 10 'test%03d.jpg'

The files will be named test001.jpg, test002.jpg etc, and the grab will stop automatically once 10 frames are captured.

Using the RPi4/3's Hardware Video Codecs

To play back the video file captured above in a window, using the Pi's hardware h264 codec, issue:

demouser@pi64 ~ $ ffplay -vcodec h264_v4l2m2m -i test.mkv

You can also use mpeg4_v4l2m2m and (if you have purchased the license from the RPF) mpeg2_v4l2m2m codecs, if your video stream is of an appropriate format.

gstreamer can be used for hardware-codec-enhanced playback too; for example:

demouser@pi64 ~ $ gst-launch-1.0 -e -vvv filesrc location=test.mkv ! matroskademux ! \
  queue ! h264parse ! v4l2h264dec capture-io-mode=4 ! \
  glimagesink

Miscellaneous Hints

You can append -nostats -loglevel 0 to any of above ffmpeg / ffplay commands, to suppress the chatty terminal output (particularly useful with the v4l2 m2m codecs).

Using MMAL

As of v1.5.2 of the image, you can also use MMAL to access the camera and codecs. So, for example, you could use:

demouser@pi64 ~ $ raspivid -v -o test.h264 -t 10000 -g 1

to capture a 10 second H264 video clip, and then:

demouser@pi64 ~ $ ffplay -vcodec h264_mmal -i test.h264

to play it back via the h/w codec. Or, of course, you could still use:

demouser@pi64 ~ $ ffplay -vcodec h264_v4l2m2m -i test.h264

to access the h/w codec via V4L2 M2M, which remains supported.