This repository contains the code that accompanies our CVPR 2019 paper Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids
You can find detailed instructions for both training your own models and using pretrained models in the examples below.
Our library has the following dependencies:
- Python 2.7
- PyTorch
- torchvision
- numpy
- scikit-learn
- progress
- pyquaternion
- trimesh
- backports.functools_lru_cache
- Cython
- sympy
- seaborn
- matplotlib
- PIL
- mayavi
They should be automatically installed by running
pip install --user -e .
In case you this doesn't work automatically try this instead,
pip install -r requirements.txt
pip install --user -e .
Please note that you might need to install python-qt4 in order to be able to
use mayavi. You can do that by simply typing
sudo apt install python-qt4
Now you are ready to start playing with the code!
For evaluating a previously trained model, we provide the forward_pass.py
script. This script performs a forward pass and predicts the parameters of the
superquadric surfaces used to represent the 3D object. With this script you can
visualize the predicted superquadrics using mayavi as well as save them as a
mesh file.
You can run it using
$ ./forward_pass.py ../demo/03001627/ /tmp/ --model_tag "dac4af24e2facd7d3000ca4b04fcd6ac" --n_primitives 18 --weight_file ../config/chair_T26AK2FES_model_699 --train_with_bernoulli --use_deformations --use_sq --dataset_type shapenet_v2
The script requires two mandatory arguments, the path to the directory that
contains the dataset, in this case it is ../demo/03001627 and the path to a
directory that will be used for saving the generated files, here /tmp. You
should also provide (even if it is not mandatory) the path to the previously
trained model via the --weight_file argument, as well as the tag of the model
you want to reconstruct (--model_tag) and the type of the dataset you are
using (--dataset_type). Note that you should provide the same arguments that
you used when training the model, regarding the configuration of the geometric
primitives (e.g number of primitives, whether or not to use superquadrics
etc.). This script automatically visualizes the predicted superquadrics using
mayavi. To save these predictions as a mesh file, simply add the
save_prediction_as_mesh argument.
Running the above command, will result in something like the following:
$ ./forward_pass.py ~/data/03001627/ /tmp/ --model_tag "dac4af24e2facd7d3000ca4b04fcd6ac" --n_primitives 18 --weight_file ../config/chair_T26AK2FES_model_699 --train_with_bernoulli --use_deformations --use_sq --dataset_type shapenet_v2
No handlers could be found for logger "trimesh"
Running code on cpu
Found 6778 'ShapeNetV2' models
1 models in total ...
R: [[-0.38369974 0.57926947 0.7191812 ]
[-0.5103006 0.5160787 -0.6879362 ]
[-0.7696545 -0.6309594 0.09758216]]
t: [[-0.01014123]
[-0.04051362]
[ 0.00080032]]
e: [1.3734075, 1.3150274]
K: [-0.33036062, 0.23313229]
R: [[-0.5362834 0.04193148 -0.8429957 ]
[ 0.78693724 0.3859552 -0.4814232 ]
[ 0.3051718 -0.92156404 -0.23997879]]
t: [[-0.18105912]
[ 0.11244812]
[ 0.09697253]]
e: [0.7390462, 1.157957]
...
R: [[ 0.9701394 -0.24186404 -0.01820319]
[-0.24252652 -0.96831805 -0.059508 ]
[-0.00323363 0.06214581 -0.99806195]]
t: [[-0.02585344]
[-0.15909581]
[-0.091534 ]]
e: [0.4002924, 0.40005013]
K: [0.41454253, 0.3229351]
0 3.581116e-08
1 0.99999857
2 0.99999917
3 3.7113843e-08
4 0.9999982
5 0.9999975
6 0.9999988
7 0.9999999
8 3.5458616e-08
9 3.721507e-08
10 3.711448e-08
11 3.9621053e-08
12 3.7611613e-08
13 0.9999976
14 0.99999905
15 0.9999981
16 0.9999982
17 0.99999785
Using 11 primitives out of 18
and a chair should appear.
To train a new network from scratch we provide the train_network.py script.
You can simply execute it by typing
$ ./train_network.py ~/data/03001627/ /tmp/ --use_sq --lr 1e-4 --n_primitives 20 --train_with_bernoulli --dataset_type shapenet_v2 --use_chamfer
Running code on cpu
Save experiment statistics in 26EKQBNTG
Found 6778 'ShapeNetV2' models
6778 models in total ...
1000/1000 [==============================] - 6s 6ms/step
Epoch 1/150 | | 15/500 - loss: 0.0110078 - pcl_to_prim: 0.0024909 - prim_to_pcl: 0.0085169 - exp_n_prims: 9.8473
You need to specify the path to the directory containing the dataset directory
as well as the path to save the generated files such as the trained models.
Note that the script automatically generates a subfolder inside the specified
output directory (in this case 26EKQBNTG), where it saves the trained models,
three .txt files with the loss evolution and a .json file with the parameters
used for the current experiment.
We also provide the visualize_sq.py script which allows you to quickly
visualize superquadrics given a set of parameters as a set of points sampled on
the surface of the superquadric surface. Please note that you need to install
python-tk to be able to use this script. You can do this my simply writing
sudo apt install python-tk.
You can simply execute it by providing your preferred shape and size parametrization as follows:
$ ./visualize_sq.py --shape 1.0,1.0 --size 0.25,0.25,0.25
Below are some example images of various superquadrics using different shape
and size parameters
Contributions such as bug fixes, bug reports, suggestions etc. are more than welcome and should be submitted in the form of new issues and/or pull requests on Github.
Our code is released under the MIT license which practically allows anyone to do anything with it. MIT license found in the LICENSE file.
Below we list some papers that are relevant to the provided code.
Ours:
By Others:
- Learning Shape Abstractions by Assembling Volumetric Primitives pdf
- 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks pdf
- Im2Struct: Recovering 3D Shape Structure From a Single RGB Image pdf
Below we also list some more papers that are more closely related to superquadrics
- Equal-Distance Sampling of Supercllipse Models pdf
- Revisiting Superquadric Fitting: A Numerically Stable Formulation link
If you found this work influential or helpful for your research, please consider citing
@Inproceedings{Paschalidou2019CVPR,
title = {Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids},
author = {Paschalidou, Despoina and Ulusoy, Ali Osman and Geiger, Andreas},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
year = {2019}
}