We present a novel neural surface reconstruction method, called NeuS (pronunciation: /nuːz/, same as "news"), for reconstructing objects and scenes with high fidelity from 2D image inputs.
Project page | Paper | Data
This is the official repo for the implementation of NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction.
The data is organized as follows:
<case_name>
|-- cameras_xxx.npz # camera parameters
|-- image
|-- 000.png # target image for each view
|-- 001.png
...
|-- mask
|-- 000.png # target mask each view (For unmasked setting, set all pixels as 255)
|-- 001.png
...
Here the cameras_xxx.npz
follows the data format in IDR, where world_mat_xx
denotes the world to image projection matrix, and scale_mat_xx
denotes the normalization matrix.
Clone this repository
git clone https://github.com/Totoro97/NeuS.git
cd NeuS
pip install -r requirements.txt
Dependencies (click to expand)
- torch==1.8.0
- opencv_python==4.5.2.52
- trimesh==3.9.8
- numpy==1.19.2
- pyhocon==0.3.57
- icecream==2.1.0
- tqdm==4.50.2
- scipy==1.7.0
- PyMCubes==0.1.2
- Training without mask
python exp_runner.py --mode train --conf ./confs/womask.conf --case <case_name>
- Training with mask
python exp_runner.py --mode train --conf ./confs/wmask.conf --case <case_name>
- Extract surface from trained model
python exp_runner.py --mode validate_mesh --conf <config_file> --case <case_name> --is_continue # use latest checkpoint
The corresponding mesh can be found in exp/<case_name>/<exp_name>/meshes/<iter_steps>.ply
.
- View interpolation
python exp_runner.py --mode interpolate_<img_idx_0>_<img_idx_1> --conf <config_file> --case <case_name> --is_continue # use latest checkpoint
The corresponding image set of view interpolation can be found in exp/<case_name>/<exp_name>/render/
.
More information can be found in preprocess_custom_data.
Cite as below if you find this repository is helpful to your project:
@article{wang2021neus,
title={NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction},
author={Wang, Peng and Liu, Lingjie and Liu, Yuan and Theobalt, Christian and Komura, Taku and Wang, Wenping},
journal={arXiv preprint arXiv:2106.10689},
year={2021}
}
Some code snippets are borrowed from IDR and NeRF-pytorch. Thanks for these great projects.