[IJCAI2023] Physics-Guided Human Motion Capture with Pose Probability Modeling

The code for IJCAI 2023 paper "Physics-Guided Human Motion Capture with Pose Probability Modeling"
Jingyi Ju, Buzhen Huang, Chen Zhu, Zhihao Li, Yangang Wang
[Paper]

Installation

Create conda environment and install dependencies.

conda create -n Physics-Guided-Mocap python=3.8
conda install pytorch torchvision torchaudio pytorch-cuda=11.1 -c pytorch -c nvidia # install pytorch
pip install -r requirements.txt

1. Due to the licenses, please download SMPL model file here.
2. The Mujoco environment is built into the project path(./mujoco_py) to avoid cumbersome environment configuration. We recommend to download mujoco-py-1.50.1.0 and mjpro150 and activation key for win10/win11.

Finally put these data following the directory structure as below:

${ROOT}
|-- assets
    |-- mujoco_models
    |-- bigfoot_template.pkl
    |-- bigfoot_template_v1.pkl
|-- data
    |-- mujoco
        |-- mujoco-py-1.50.1.0
        |-- mjpro150
        |-- mjkey.txt
    |-- sample_data
        |-- amass_copycat_occlusion.pkl
        |-- amass_copycat_take5_test_small.pkl
        |-- standing_neutral.pkl
    |-- smpl
        |-- SMPL_NEUTRAL.pkl
        |-- SMPL_FEMALE.pkl
        |-- SMPL_MALE.pkl
    |-- init_data.pkl
    |-- iter_19000.p
    |-- J_regressor_h36m.npy
    |-- J_regressor_halpe.npy
    |-- J_regressor_lsp.npy

Usage

Demo

• Reconstruct physically-plausible human motions with physics-guided diffusion framework.

python demo.py

TODO:

• Release trained model
• Release training code

Citation

If you find this code useful for your research, please consider citing the paper.

@inproceedings{ijcai2023p105,
  title     = {Physics-Guided Human Motion Capture with Pose Probability Modeling},
  author    = {Ju, Jingyi and Huang, Buzhen and Zhu, Chen and Li, Zhihao and Wang, Yangang},
  booktitle = {International Joint Conference on Artificial Intelligence, IJCAI},
  pages     = {947--955},
  year      = {2023},
}

Reference

Some of the code is based on the following works. We gratefully appreciate the impact they have on our work.

UniversalHumanoidControl
SMPL-X
CLIFF