Benchmarks: Raspberry Pi

This guide shows how to setup and run performance benchmarks on a Raspberry Pi distributed server.

There are two types of benchmark:

• Hardware performance tests: those which run entirely on the server typically to test the SDcard storage
• Real-world simulation tests: These typically run on a workstation, which is networked with the Raspberry Pi

Setting up the server hardware

Initialize an SD card with the Raspian image.

Instructions for download and burn can be found here

(Optional) set a fixed IP and hostname now

It is helpful to fix the IP address and hostname of the distributed server.
To pre-configure this information, mount the card on the local computer and edit the following files. IMPORTANT: Make sure these files are edited on the mounted SD card, and not on the /etc of the host computer!

/etc/hostname change raspberrypi to the new hostname

/etc/hosts 127.0.1.1 newhostname <-- change raspberrypi to the new hostname

/etc/network/interfaces change the eth0 section to this (change values to suit your local configuration):

iface eth0 inet static hostname newhostname address 192.168.xxx.xxx netmask 255.255.255.0 gateway 192.168.xxx.xxx

Initial setup in Raspberry pi

Insert the new card, plugin a wired ethernet connection and switch on

ssh pi@rasp.pi.ip.address password is raspberry

# set 16mb video memory sudo sh -c 'echo "gpu_mem=16" >>/boot/config.txt'

Finally, expand the root filesystem

sudo raspi-config select option 1

sudo reboot

Run the setup script

cut/paste this script into a temporary .sh file and execute with bash (this takes about 10 mins to run)

## update os
sudo apt-get -y update 
sudo apt-get -y upgrade 
## add packages
sudo apt-get -y install python-m2crypto 
sudo apt-get -y install git-core
sudo apt-get -y install sysstat
## cleanup
sudo apt-get -y autoremove
## install KA-Lite software
cd ~
git clone https://github.com/learningequality/ka-lite.git
cd ka-lite/
## switch to develop branch
git checkout develop
## install KA-Lite
python kalite/manage.py setup --username="admin" --password="admin" --noinput
## create a facility
kalite/manage.py generaterealdata facility
## set ka-lite to autostart on boot
sudo scripts/runatboot.sh
## install the optimizations
scripts/optimizerpi.sh

Finally

Reboot the Raspberry Pi and check connectivity by browsing to:

http://your.rasp.ip.address:8008

Hardware performance tests

Preparation

Hardware tests will make repeated IO attempts on the KA-Lite database. Before running Hardware performance tests, the database must be populated with exercise and video logs. An easy way to do this is with a generator:

kalite/manage.py benchmark generatedata

Average elapsed (0): 1871.54885006 Exiting (0)

The elapsed time can be used to calculate the database insert rate.

Database insert rate

If data was created with generatedata, the total elapsed seconds would have been reported. Simply divide the total inserted by the elapsed time taken by generatedata. To discover the number of records inserted, do:

kalite/manage.py shell_plus

>>> print "total inserted = ", str( 21+ VideoLog.objects.count() + ExerciseLog.objects.count() )

1000 random reads

This test makes 1000 random selects from the database

kalite/manage.py benchmark 1000reads --iterations=2

Running 1000reads 2 times in 1 clients
Starting (0)
Average elapsed (0): 33.5336028337
Exiting (0)

100 random updates

This test makes 1000 random selects from the database

kalite/manage.py benchmark 100updates

Running 100updates 1 times in 1 clients
Starting (0)
Average elapsed (0): 22.6799740791
Exiting (0)

Simulation tests

Preparation

A client machine is needed to run the selenium scripts. Specification:

• Linux/Mac with at least 4 gig of memory
• Firefox/Iceweasel browser installed
• KA-Lite software installed
• On the same network as the Raspberry Pi distributed server

Analyzing performance

The simulation tests will report activity from a client viewpoint, but cannot report server-side resources used.

Therefore, it is necessary to monitor the Raspberry Pi performance separately. A simple way is to snapshot the situation before a test begins and and again after a test has been run.

• To show IO activity counters, use the command iostat
• To show CPU of the python webserver, use the command ps -eo cputime,args |grep cherrypy

Selenium Student

This is a basic test which runs exercises. Videos are NOT watched in this test, so video content is not needed.

This simulation is repeatable - the simulation will follow identical paths if run again.

IMPORTANT NOTES:

• Student accounts are created during simulation tests, but often fail to complete properly
• To overcome this issue, repeat the test multiple times, until all student accounts have been created
• Starting too many clients will overload the client machine and generate random python errors

./kalite/manage.py benchmark seleniumstudent --url=http://rasp.pi.IP.address:8008 --clients=4 --duration=2

Running seleniumstudent 1 times in 4 clients
Starting (0)
Starting (1)
Starting (2)
Starting (3)
(0)waiting until it's time to start.
(1)waiting until it's time to start.
(2)waiting until it's time to start.
(3)waiting until it's time to start.
(benchmark_0) exercises completed = 3
Average elapsed (0): 124.013151169
Exiting (0)
(benchmark_2) exercises completed = 7
Average elapsed (2): 122.644128084
Exiting (2)
(benchmark_3) exercises completed = 4
Average elapsed (3): 126.885775089
Exiting (3)
(benchmark_1) exercises completed = 5
Average elapsed (1): 130.957086086
Exiting (1)