Learning Predict-and-Simulate Policies From Unorganized Human Motion Data

Abstract

This code implements the paper "Learning Predict-and-Simulate Policies From Unorganized Human Motion Data". With this system, users can control characters interactively in physics-based environment.

Publications

Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee., Learning Predict-and-Simulate Policies From Unorganized Human Motion Data, ACM Trans. Graph. 38, 6, (SIGGRAPH Asia 2019)

Project Page : http://mrl.snu.ac.kr/publications/ProjectICC/ICC.html

Youtube : https://youtu.be/9dgUMli0HFU

Paper : http://mrl.snu.ac.kr/publications/ProjectICC/ICC.pdf

Blog

http://mrl.snu.ac.kr/blog/ProjectICC

How to install

OS

We checked that the code works in Ubuntu 16.04 and 18.04.

Install dependencies

sudo apt-get install -y git python3-dev python3-pip
sudo apt-get install -y python3-tk
pip3 install ipython
pip3 install matplotlib
sudo apt-get install -y curl
sudo apt-get install -y libglew-dev

# dart dependencies
sudo apt-get install -y build-essential cmake pkg-config
sudo apt-get install -y libeigen3-dev libassimp-dev libccd-dev libfcl-dev libboost-regex-dev libboost-system-dev
sudo apt-get install -y libopenscenegraph-dev
sudo apt-get install -y libnlopt-dev
sudo apt-get install -y coinor-libipopt-dev
sudo apt-get install -y libbullet-dev
sudo apt-get install -y libode-dev
sudo apt-get install -y liboctomap-dev
sudo apt-get install -y libflann-dev
sudo apt-get install -y libtinyxml2-dev
sudo apt-get install -y liburdfdom-dev
sudo apt-get install -y libxi-dev libxmu-dev freeglut3-dev
sudo apt-get install -y libopenscenegraph-dev

Install Boost

echo "downloading boost 1.66.0"
wget https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.gz
tar -xzf ./boost_1_66_0.tar.gz
rm ./boost_1_66_0.tar.gz

echo "installing boost"
cd ./boost_1_66_0
./bootstrap.sh --with-python=python3 --with-libraries=atomic,chrono,filesystem,python,system,regex
sudo ./b2 --with-python --with-filesystem --with-regex --with-system install
cd ..

Install DART

Please refer to http://dartsim.github.io/

git clone git://github.com/dartsim/dart.git
cd dart
git checkout tags/v6.7.0
mkdir build
cd build
cmake ..
make -j4
sudo make install

Install this repository

git clone https://github.com/snumrl/ICC.git
cd ICC
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4

How to run

Data

Our system requires augmented motion data generated by Interactive Character Animation(this code will be public soon). As a test example, we provide chicken-hopping motion data set and its trained networks.

Compile and Run

• Configurations Configurations for training and simulation are defined in configurations/*.xml.(e.g. configurations/chicken.xml)

• Train motion generator

cd policy
python3 train_mg.py -m chicken

• Train tracking controller

cd policy
# -s : session name, output networks will be stored in output/{session_name}
# -c : configurations file path
python3 train.py -s chicken -c ../configurations/chicken.xml

• Train tracking controller again

cd policy
# -s : session name, output networks will be stored in output/{session_name}
# -c : configurations file path
# -n : initial network file path
# -t : initial network type
#  None : latest network
#  smax : network whose step is max
#  rmax : network whose reward is max
python3 train.py -s chicken_1 -c ../configurations/chicken.xml -n ../output/chicken -t smax

• Run result

cd policy
# ../build/interactive/interactive path type
# path : network file path
# type : network type, None, smax, rmax
#  None : latest network
#  smax : network whose step is max
#  rmax : network whose reward is max
../build/interactive/interactive ../output/chicken_1 smax

Additional run files

• Play tracking controller in fixed trajectory

cd policy
python3 play.py -n ../output/chicken -t smax

Output results are written as files in ../output/play/*.rec.

• Follow fixed trajectory

cd policy
python3 follow.py -c ../configurations/chicken.xml

Output results are written as files in ../output/follow/*.rec.

• Render *.rec file

cd policy
../build/render/render ../output/play/0.rec