en_Installation_Pynq

IkaLog on Digilent Pynq board (Project Inq)

Pynq is the evaluation platform of Xilinx Zynq APSOC(All Programmable System on Chip). PYNQ-Z1 is the available evaluation board with dual-core ARM processors, 512MB DDR3 Memory, and 2X HDMI Ports. Xilinx also provides Linux image that can be written to micro SD card and boot the board easily.

IkaLog is now functionable with PYNQ-Z1 (as POC.)

Why PYNQ-Z1?

Generally, IkaLog needs these goods to run:

• A computer (Windows, Mac, and Linux) to run IkaLog
• An HDMI Capture Device supported by corresponding operating system
• An HDMI video splitter to feed the video to the computer, as well as your TV
• Cables to connect these devices
• And, Wii U console, of course

To obtain these, you may need to spend $300 or more. If you don't have a computer that works with IkaLog, additional thousands of dollars will be spent.

Project Inq aims to run IkaLog on an inexpensive computer.

We do know Raspberry Pi 3 is available today, but video input features are lacking if you want to run IkaLog on the board. Of course, you still need to get a splitter.

Digilent PYNQ-Z1 board has everything we need to run IkaLog. It has dual-core ARM Cortex A9 processor that show better performance compared to RPi 3(on IkaLog workload), and HDMI Input/Output that we have an opportunity to configure those ports as HDMI video capture device and splitter. PL also can be configured as some algorithm accelerators.

Project Inq is still Proof of Concept.

How to run IkaLog on PYNQ-Z1

This procedure is subject to change.

• Write Linux image to a micro SD card. pynq_z1_image_2016_09_14.img should work.
• Insert the micro SD card into Pynq, set the jumper to SD mode. Linux should come up.
• Login to Pynq using id/pass = xilinx/xilinx. The default IP address is 192.168.2.99/24.
• Install dependencies.

pip3 install u-msgpack-python
...

• Clone IkaLog from GitHub.

https://github.com/hasegaw/IkaLog.git

• Configure IkaConfig.py.
  Since Pynq doesn't have X Window system, you should disable Screen output. Source PYNQ is available.

INPUT_SOURCE='PYNQ'

• PYNQ Input utilizes base.bit overlay used by Jupiter Notebook demonstration. Make sure loading the overlay before running IkaLog. Run python and do (as root):

from pynq import Overlay
Overlay("base.bit").download()

• Run IkaLog.py (as root; IkaLog needs to access to HDMI interface)

python3 IkaLog.py

No! I don't have Wii U here!

dhcp44-82:IkaLog_github hasegaw$ wget https://dl.dropboxusercontent.com/u/14421778/IkaLog/ikalog_sample.mp4
dhcp44-82:IkaLog_github hasegaw$ python3 IkaLog.py -f ikalog_sample.mp4

Build OpenCV with additional HAL

OpenCV 3.1 included in PYNQ doesn't utilize NEON instructions. You need to recompile OpenCV to maximize IkaLog performance. In our test, NEON-enabled OpenCV performs 2X throughput in color conversion functions.

  312  git clone --depth 1 https://github.com/Itseez/opencv.git
  313  git clone --depth 1 https://github.com/Itseez/opencv_contrib.git
  314  cd opencv
  315  cd build/
  316  cmake -D CMAKE_INSTALL_PREFIX=$HOME/local/ -D CMAKE_BUILD_TYPE=RELEASE \
      　　　　　　-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
      　　　　　　-D BUILD_opencv_python3=ON -D BUILD_opencv_python2=OFF \
      　　　　　　-D WITH_CUDA=OFF -D WITH_NEON=ON -D BUILD_EXAMPLES=OFF ..
  317 make -j3 build && make install

The build process takes several hours to complete. Run the build before you sleep.

cat << EOF >> ~/.bash_profile (or somewhere) 
export LD_LIBRARY_PATH=$HOME/local/lib/:$HOME/local/lib/python3.4/dist-packages/
export PYTHONPATH=.:$HOME/local/lib/python3.4/dist-packages/
EOF

Build IkaMatcher HAL for IkaLog

On some branches we have proof of concept of NEON acceleration (though we have more overhead on Python ctypes layer right now)

cd lib
make

How to switch HAL used by IkaMatcher2?

See https://github.com/hasegaw/IkaLog/blob/940e797ef2d1cf8bdde4f4ea20ebbdfa18582fe1/ikalog/utils/ikamatcher2/matcher.py#L29

How to profile IkaLog?

python3 -m cProfile -s tottime IkaLog.py -f ikalog_sample.mp4 | tee output.txt

The opportunity of HW H.264 decoder on Raspberry Pi

If you are trying to process video files on the filesystem, cv2.VideoCapture.read() functions will be the significant bottleneck. It shows 50% of the runtime is consumed by H.264 decoder.

In Raspberry Pi, HW H.264 decoder can be enabled by separately sold IP license. The hardware decoder seems to be work with GStreamer. Since we already support GStreamer source, IkaLog should be able to offload the decode process to HW decoder on Raspberry Pi.

See our Linux page in this wiki to get IkaLog works with GStreamer. This is still an idea and not proven.

Consideration about PYNQ default bitstream

TBD

Have a fun and hack!

Any feedbacks, pull requests are appreciated. If you are going to work on Project Inq, please contact hasegaw join IkaLog slack to maximize our bandwidth and collaborate.