Introduction

Humanoid explores Android apps like human. It uses deep learning techniques to borrow experiences from app usage traces generated by human.

Currently Humanoid works with DroidBot. When DroidBot explores an Android app in model-based policy, it will generate several possible input events according to current UI state. Humanoid than sort the events such that events that will be performed by human most likely will be fired first.

Prerequisite

• Python 3.x
• Tensorflow
• DroidBot
• PyFlann (one may need to make it work under Python 3 according to its README)

Needed only if one wants to train Humanoid model from scratch:

• RICO dataset

Quick Start

NOTES BEFORE USAGE: When used with DroidBot, Humanoid supports 1280x720, 1920x1080 and 2560x1440 screens only. Also, the Android navigation bar should be turned on.

Humanoid is an XMLRPC service which can be shared among multiple DroidBot instances.

First, start Humanoid service by

$ python3 agent.py -c config.json

After seeing

=== Humanoid RPC service ready at localhost:50405 ===

one can start DroidBot instances with -humanoid localhost:50405 parameter. Now DroidBot will make use of Humanoid model when using model-based policies, such as dfs_greedy. See DroidBot README page for more usage details.

Modify domain and port values in config.json in the same directory to deploy Humanoid service at other addresses.

How to setup Humandroid (editted by Themis)

1. Humanoid uses TensorFlow 1.12 (see this issue). However, TensorFlow 1.12 is supported by Python 3.5, 3.6 and 3.7 (see this issue). Thus, if your Python version is newer than these versions, you should install old Python version, e.g., Python 3.6 (follow this link).

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.6

2. create python virtual environment via virtualenv and install tensorflow 1.12

What is virtualenv? (https://help.dreamhost.com/hc/en-us/articles/115000695551-Installing-and-using-virtualenv-with-Python-3)

Install virtualenv

apt-get install python3-virtualenv
cd Humanoid-tool
virtualenv -p /usr/bin/python3.6 venv

Install TensorFlow (these two links may provide some guidance: link1, link2)

pip install -U tensorflow==1.12

3. install additional tool and moudles

Humanoid relies on DroidBot. Thus, you need to setup DroidBot. Please use our version of DroidBot (where we have fixed several issues): https://github.com/the-themis-benchmarks/droidbot/tree/themis-branch

git clone git@github.com:the-themis-benchmarks/droidbot.git
cd droidbot
pip install -e .

pip install matplotlib
pip install scipy
pip install pyflann-py3