Skip to content

Latest commit

 

History

History
124 lines (83 loc) · 5.21 KB

mpfs-video-kit-h264-modular-pipeline-demo.md

File metadata and controls

124 lines (83 loc) · 5.21 KB
Raw

PolarFire SoC Video Kit H.264 Modular Pipeline Demo

This document describes about H.264 modular pipeline.

Table of Contents

Overview

The H.264 modular pipeline has been developed using the Linux Kernel Media Framework, which complements the Video for Linux (V4L2) Framework.

The media framework requires the topology of the pipeline to be described using a topology graph. A topology graph consists of entities and pads. An entity can be any logical or physical functional block. Pads enable connections between entities. Pads can be source or sink pads, in which the source pad of an entity gets connected to the sink pad of the compatible entity.

An end-user application can access and configure the above graph using media device and sub-devices nodes, through the libmediactl and libv4l2subdev libraries respectively.

The Media Framework provides APIs to achieve the above functionality. It includes:

  • enumeration of entities and pads.
  • setting format and resolution.

H.264 Modular Pipeline

  • The H.264 modular pipeline consists of the following IP's (each IP accessible through the corresponding drivers)

Imx334 camera --> MIPI CSI2 IP --> GVPC (MONO to RGB) --> Image Enhancement --> Scaler --> OSD --> GVPC (RGB to YUV) --> H264 -->Video DMA IP (/dev/video0)

  • Each IP core driver registers itself as an entity with two pads (source/sink).
  • Output of one IP core is fed to the input of next IP core. All parameter values are also passed on from the previous to the next IP core.
  • Video DMA-write driver is responsible for creating v4l2 sub-devices and media devices.
  • IP cores and corresponding drivers can be inserted or removed as required.

Block Diagram

The following figure shows the Fabric (IP) -- MSS (Processor) Interface: The control path is through AXI-Lite Interface.

Software Stack

Linux will load all v4l2 modular drivers based on the video pipeline specified in the device tree.

The following figure shows the software stack for H.264 video pipeline.

The framework exposes the following device node types to user space to control certain aspects of the pipeline:

  • Media device node: /dev/media*
  • Video device node: /dev/video*
  • V4L subdevice node: /dev/v4l-subdev*

Where/dev/video* nodes are responsible for capturing the images, /dev/v4l-subdev* nodes are responsible for corresponding IP controls and /dev/media* nodes are for changing the resolution and format

In the above diagram, the Video device node is created by the Video Capture (DMA) Driver, while the subdevice nodes get created by the remaining drivers.

Pre-Requisites

  • These pre-requisites are in addition to the ones mentioned in h264 demo prerequisites
  • Make sure that the H264 modular pipeline design (MPFS_VIDEO_KIT_H264_MM_DESIGN_xxxx_xx.zip) is programmed to the PolarFire SoC Video Kit. A FlashPro Express programming job file can be found in the Video Kit reference design.

Steps To Load H.264 Modular Pipeline

  • Note: If FlashPro Express programming job file (taken from Video Kit reference design) version is v2024.06 or above, there is no need to follow the below steps, as the devicetree overlay for the modular pipeline will be loaded from flash automatically while booting Linux.

  • To load the H.264 modular pipeline, stop at U-Boot prompt (while booting on boot console press any key on key board to stop, U-Boot logs looks like below):

CPU:   rv64imafdc
Model: Microchip PolarFire-SoC Video Kit
DRAM:  1 GiB (effective 2.8 GiB)
Core:  49 devices, 11 uclasses, devicetree: separate
MMC:   mmc@20008000: 0
Loading Environment from FAT... OK
In:    serial@20100000
Out:   serial@20100000
Err:   serial@20100000
Net:   eth0: ethernet@20110000
Hit any key to stop autoboot:  0
RISC-V #
  • Run the commands below:
RISC-V # load mmc 0:1 ${scriptaddr} fitImage;
RISC-V # bootm start ${scriptaddr}#conf-microchip_mpfs-video-kit.dtb#conf-mpfs_modular_h264_pipeline.dtbo;
RISC-V # bootm loados ${scriptaddr};
RISC-V # bootm ramdisk;
RISC-V # bootm prep;
RISC-V # fdt set /soc/ethernet@20112000 mac-address ${videokit_mac_addr0};
RISC-V # fdt set /soc/ethernet@20110000 mac-address ${videokit_mac_addr0};
RISC-V # bootm go;

Running the Demo

Please see the Running the Demo guide for information on how to run the modular pipeline demo.