DA Wand [Project Page]
Code for "DA Wand: Distortion-Aware Selection using Neural Mesh Parameterization" [CVPR 2023]
# With GPU
conda env create --file dawand.yml
conda activate dawand
# Without GPU
conda env create --file dawand_cpu.yml
conda activate dawand
To install pygco, go into the gco_python
folder and run make
, then run python setup.py install
. If you experience any issues please refer to the gco_python repo.
- Python 3.7
- CUDA 11
- 16 GB GPU (for training)
- Blender 3.3.1 (for synthetic dataset generation)
Distortion Self-Supervision Dataset
To train, simply provide the path to the dataset folder as inputs to the argument --dataroot
and the path to the test subfolder to the argument --test_dir
. Set --gpu_ids
to -1 if training without a GPU. See the scripts
folder for example commands for training the network with the same parameters as in the paper.
Call the script in ./scripts/generate_synthetic_data.sh
to generate the synthetic dataset using the procedure outlined in the paper. The data will be stored in ./datasets/synthetic_data
.
To generate the deformed primitives with custom parameters, you will need to have Blender 3.3.1 installed and available in PATH. From there, you can generate random deformed primitives using the primitives provided in datasets/primitives
by calling
blender --background --python blender_deform.py --datadir ./datasets/primitives --outdir ./datasets/deformed_primitives ...
This will generate a set of deformed primitives with the same ground truth near-developable decompositions. Refer to the blender_deform.py
file for the adjustable parameters for generating the synthetic data.
After generating the deformed primitives, you can call generate_synthetic_data.py
with custom parameters to construct the synthetic dataset with randomly sampled selection points and ground truth labels.
The differentiable parameterization layer enables training DA Wand over any arbitrary set of meshes. Creating a dataset for distortion self-supervised training is simple.
- Create a folder with the name of the dataset, with subfolders
train
andtest
Split your training meshes between thetrain
andtest
folders. - For each mesh in the subfolders, sample selection points as desired into a Python list and use
dill
to save them as pickled files in a folder titledanchors
. The name of the pickled files should be{name of obj}.pkl
. There should be separatetrain/anchors
andtest/anchors
folders. - To incorporate mixed training with ground truth labels, copy the respective labelled meshes into the
train
andtest
folders, copy their anchor.pkl
files into theanchors
subfolder, and create a new folder titledlabels
to store the labels. Each label file should be a binary numpy array with the same length as the number of mesh triangles. The label files should be titled by{name of obj}{anchor index}.npy
, where the anchor index is the index of the respective selection point from the mesh selection point list saved inanchors
. To train with mixed data, simply pass the flags--supervised --mixedtraining
into thetrain.py
call.
demo.mp4
The interactive demo was created using Polyscope. To run the demo, run the following command
python interactive.py --modeldir ./checkpoints --modelname dawand --meshdir {path to obj file} --meshfile {name of obj file}
To use a different DAWand model, simply change the inputs to modeldir
and modelname
. The application will expect the model weights to be saved with the naming convention {modelname}_net.pth
Note: DA Wand is trained on meshes within a restricted edge range (5000 - 12,000), and will be mostly effective on meshes within that range. For meshes of lower or higher resolution, we recommend either remeshing or retraining the model to meet the desired the resolution.
The implementation code for DA Wand relies on a number of excellent public source projects for geometry processing, analysis, and visualization. These projects are acknowledged below:
@article{liu2022dawand,
author = {Liu, Richard and Aigerman, Noam and Kim, Vladimir G. and Hanocka, Rana},
title = {DA Wand: Distortion-Aware Selection using Neural Mesh Parameterization},
journal = {arXiv},
year = {2022}
}