Skip to content

Latest commit

 

History

History
202 lines (145 loc) · 6.72 KB

TRAIN_EVAL.md

File metadata and controls

202 lines (145 loc) · 6.72 KB
Raw

Training and Evaluation

Table of Contents

Dataset Preparation

cd /path/to/this/repo
export OPLANES_ROOT=$PWD

Download Raw Data

OPlanes are trained on SAIL-VOS 3D (S3D). Please download the dataset following instructions on the website.

Place the downloaded dataset at ${OPLANES_ROOT}/datasets/s3d/raw.

Install ManifoldPlus

We use ManifoldPlus to create a watertight mesh from the raw mesh. Please follow the official instruction to install the package.

You can install it anywhere you prefer. Here we install it to ${OPLANES_ROOT}/external for illustartion purpose:

cd ${OPLANES_ROOT}/external

git clone https://github.com/hjwdzh/ManifoldPlus.git

export ManfioldPlusRoot=${OPLANES_ROOT}/external/ManifoldPlus

cd ${ManfioldPlusRoot}
git submodule update --init --recursive

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j8

Generate Binvox Files

Since it is time-consuming to generate binvox files for all meshes, here we use train_dryrun.txt and val_dryrun.txt to illustrate how to run the command.

Please replace them with train_27588.txt and val_4300.txt for full training and evaluation.

The generated binvox files will be saved in ${OPLANES_ROOT}/datasets/s3d/binvox_256.

cd ${OPLANES_ROOT}

# for train split
conda activate oplanes && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
python ${OPLANES_ROOT}/oplanes/data/create_binvox.py \
--manifoldplus_exe_f ${ManfioldPlusRoot}/build/manifold \
--process_f ${OPLANES_ROOT}/datasets/s3d/splits/train_dryrun.txt \
--data_root ${OPLANES_ROOT}/datasets/s3d/raw \
--save_root ${OPLANES_ROOT}/datasets/s3d/binvox_256

# for val split
conda activate oplanes && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
python ${OPLANES_ROOT}/oplanes/data/create_binvox.py \
--manifoldplus_exe_f ${ManfioldPlusRoot}/build/manifold \
--process_f ${OPLANES_ROOT}/datasets/s3d/splits/val_dryrun.txt \
--data_root ${OPLANES_ROOT}/datasets/s3d/raw \
--save_root ${OPLANES_ROOT}/datasets/s3d/binvox_256

Training

The following command will train OPlanes models. If you have generated full binvox files in the above step, set --dryrun 0 instead.

Checkpoints will be saved in ${OPLANES_ROOT}/exps.

conda activate oplanes && \
export NCCL_P2P_DISABLE=1 && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
python ${OPLANES_ROOT}/oplanes/run.py \
--config_f ${OPLANES_ROOT}/configs/default.yml \
--dryrun 1 \
--log 1

Evaluation

Install ONet

We utilize Occupancy Network (ONet)'s utility functions. Therefore, we need to install ONet.

For ONet's utility functions, we modify two things:

  1. We comment out the specific requirements for GPU architecture in libfusiongpu/CMakeLists.txt such that it can be compiled against more GPU categories.
  2. Following suggestions from this answer, we modify libfusiongpu/cyfusion.cpp, librender/pyrender.cpp, libmcubes/mcubes.cpp.

Please follow the official instructions to install it:

# setup ONet
cd ${OPLANES_ROOT}/external/onet
python setup.py build_ext --inplace

cd ${OPLANES_ROOT}/external/onet/external/mesh-fusion

# build pyfusion
# use libfusioncpu alternatively!
cd libfusiongpu
mkdir build
cd build
cmake ..
make
cd ..
python setup.py build_ext --inplace

cd ..
# build PyMCubes
cd libmcubes
python setup.py build_ext --inplace

Sample GT Points

We sample ground-truth (GT) points and save them to disk for fast evaluation.

Points and information for evaluation will be saved in ${OPLANES_ROOT}/datasets/s3d/gt_pts_for_eval.

cd ${OPLANES_ROOT}

conda activate oplanes && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
export NCCL_P2P_DISABLE=1 && \
python ${OPLANES_ROOT}/oplanes/eval/sample_gt_points.py \
--nproc 1 \
--data_root ${OPLANES_ROOT}/datasets/s3d/raw \
--save_root_dir ${OPLANES_ROOT}/datasets/s3d/gt_pts_for_eval \
--bin_f ${ManfioldPlusRoot}/build/manifold \
--split_f ${OPLANES_ROOT}/datasets/s3d/splits/val_dryrun.txt \
--is_1st_time 1

Please replace val_dryrun.txt with val_4300.txt for full evaluation.

Generate Mesh for S3D

We first generate meshes for S3D.

Please set --ckpt_f to a checkpoint file you prefer. Here we use a pretrained checkpoint for the illustration purpose.

Generated meshes will be saved in a folder eval in the same directory as the checkpoint file. In this example case, it is ${OPLANES_ROOT}/ckpts/pretrained/seed_0/eval.

Note, we do not use any post-processing when evaluting for quantitative results. Namely, we set --smooth_mcube 0.

cd ${OPLANES_ROOT}

conda activate oplanes && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
python ${OPLANES_ROOT}/oplanes/eval/gen_mesh.py \
--nproc 1 \
--data_root ${OPLANES_ROOT}/datasets/s3d/raw \
--ckpt_f ${OPLANES_ROOT}/ckpts/pretrained/seed_0/checkpoints/model-oplanes-s3d-epoch=14-val_acc=0.9204.ckpt \
--smooth_mcube 0 \
--n_bins 256 \
--given_crop_info_root ${OPLANES_ROOT}/datasets/s3d/gt_pts_for_eval \
--split_f ${OPLANES_ROOT}/datasets/s3d/splits/val_dryrun.txt

Please replace val_dryrun.txt with val_4300.txt for full evaluation.

Run Evaluation

Results will be saved in ${OPLANES_ROOT}/ckpts/pretrained/seed_0/eval/results/.

A binary file eval_dicts_all.pt stores all the results. You can load it as joblib.load(eval_dicts_all.pt).

conda activate oplanes && \
export PYTHONPATH=${OPLANES_ROOT}:$PYTHONPATH && \
python ${OPLANES_ROOT}/oplanes/eval/eval_meshes.py \
--nproc 1 \
--ckpt_f ${OPLANES_ROOT}/ckpts/pretrained/seed_0/checkpoints/model-oplanes-s3d-epoch=14-val_acc=0.9204.ckpt \
--n_bins 256 \
--gt_dir ${OPLANES_ROOT}/datasets/s3d/gt_pts_for_eval \
--split_f ${OPLANES_ROOT}/datasets/s3d/splits/val_dryrun.txt

Please replace val_dryrun.txt with val_4300.txt for full evaluation.