- OS: Ubuntu 18.04
- NVIDIA GPU with CUDA>=10.1
- Clone this repo by
git clone git@github.com:Freedomcls/NeRF-VPT.git
- Python>=3.6 (installation via anaconda is recommended, use
conda create -n nerf_vpt python=3.6
to create a conda environment and activate it byconda activate nerf_vpt
) - Python libraries
- Install core requirements by
pip install -r requirements.txt
- Install
torchsearchsorted
bycd torchsearchsorted
thenpip install .
- Install core requirements by
Download data from here.
Baidu Netdisk, extraction code: 1234
Unzip the data and place it in the current directory.
First, it is necessary to train NeRF as Stage 0.
sh run_stage0.sh
Render images of the training, validation, and test datasets from the corresponding viewpoints as view prompts.
sh run_stage0_render.sh
This script will output the PSNR metric of the model on the dataset, and the rendered images will be saved in results/replica/replica_stage0_xx
.
Proceed with training the NeRF-VPT model next.
sh run_multi_stage.sh
Render images of the training, validation, and test datasets from the corresponding viewpoints as view prompts in Stage 1~N.
sh run_mul_stage_render.sh
You can continue the next stage of training using run_multi_stage.sh
, but you would need to modify the corresponding parameters within it.
For example, if training stage2, you need to modify --exp_name replica_stage1
to --exp_name replica_stage2
.
The same in inference phase, you need to modify:
1.all the --scene_name replica_stage1_xxx
to --scene_name replica_stage2_xxx
;
2.all the --ckpt_path ckpts/replica_stage1/xxx
to --ckpt_path ckpts/replica_stage2/xxx
;
3.all the --prompt_path results/replica/replica_stage0
to --prompt_path results/replica/replica_stage1
.
Also, the script will output the PSNR metric of the model.