Skip to content

Latest commit

 

History

History
125 lines (90 loc) · 4.15 KB

README.md

File metadata and controls

125 lines (90 loc) · 4.15 KB
Raw

HelixSurf: A Robust and Efficient Neural Implicit Surface Learning of Indoor Scenes with Iterative Intertwined Regularization

Zhihao Liang, Zhangjin Huang, Changxing Ding, Kui Jia

CVPR 2023

Paper | Arxiv | Project Page

Requirements

  • Python 3.8.13
  • PyTorch 1.11.0+cu113
  • CUDA 11.3
  • Prim3D for efficient RayCasting and MarchingCubes
  • gmvs for efficient PatchMatch MVS
pip install -r requirements.txt

Installation

git submodule update --init --recursive
python setup.py develop

Prim3D and gmvs can be obtained from submodules. Please follow their README.md and install them.

Training

take 0616_00 in ScanNet as an example

  • step0: Download our processed data from [Google Drive] [Baidu Cloud]（提取码：a6sa），unzip HelixSurf_data.zip as HelixSurf_data under the project root directory.
HelixSurf_data
  ├── scene_data # stores our processed data
  └── mvs_results # stores our MVS results

for custom data, please refer to ManhattanSDF to generate data.

  • step1 (Optional): Fix the parameters in run_scripts/first_mvs.sh and run the follow command to perform mvs:

you can also use the our mvs results from HelixSurf_data/mvs_results and skip this step

sh run_scripts/first_mvs.sh
  • step2: Train the first epoch
sh run_scripts/0616_train.sh mvs "--casting"
  • step3: Perform the second mvs
sh run_scripts/next_mvs.sh
  • step4: Train the second epoch
sh run_scripts/0616_train.sh mvs_from_1epoch "--load_ckpt ckpt/0616_00_default/ckpt_1.pth --consistant -im_psize 11"

Training using pretrained geometric cues

take 0616_00 in ScanNet as an example we provide the run_script for 0616_00. You can just modify this script slightly and get the training launch script for other scene. You can also refer MonoSDF to generate geometric cues using Omnidata. The

  • step0: prepare original data
  • step1: generate geometric cues (just for normal)
python scripts/pretrained_geometric.py --task normal \
  --img_path HelixSurf_data/scene_data/0616_00/images/ \
  --output_path HelixSurf_data/scene_data/0616_00/pretrained \
  --omnidata_path $OMNIDATA_PROJECT/omnidata_tools/torch \
  --pretrained_models $OMNIDATA_PROJECT/omnidata_tools/torch/pretrained_models/
  • step2: training
sh run_scripts/0616_pretrained_train.sh mvs "--casting"

Inference

Run the follow command to perform inference:

python scripts/inference.py --config configs/default.yaml --data_dir $DATA_DIR --scene 0616_00 --ckpt $CKPT_PATH

We provide pretrained models. [Google Drive] [Baidu Cloud]（提取码：yil8）.

TODO

  • Pretrained model support (normal cues)
  • Unbiased depth expectation for regularization

Acknowledge

Citation

If you find this work useful in your research, please cite:

@misc{liang2023helixsurf,
      title={HelixSurf: A Robust and Efficient Neural Implicit Surface Learning of Indoor Scenes with Iterative Intertwined Regularization}, 
      author={Zhihao Liang and Zhangjin Huang and Changxing Ding and Kui Jia},
      year={2023},
      eprint={2302.14340},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}