Project Page | Paper | Data
Official PyTorch implementation of Pandora: an inverse rendering technique that exploits neural implicit representations and polarization cues.
PANDORA: Polarization-Aided Neural Decomposition Of Radiance
Akshat Dave, Yongyi Zhao, Ashok Veeraraghavan
Computational Imaging Lab, Rice University
accepted for ECCV 2022
Create a new Anaconda environment using the supplied environment.yml
conda env create -f environment.yml
Unzip this zip file (4.5 GB) into the data
folder of the repo directory. The zip file contains real and rendered multi-view polarimetric datasets shown in the paper.
Refer to dataio/Ours.py
and data/Mitsuba2.py
for pre-processing of real and rendered data respectively.
Run the following command to train geometry and radiance neural representations from multi-view polarimetric images.
python -m train --config configs/real_ceramic_owl.yaml
Config files input through --config
describe the parameters required for training. As an example the parameters for real ceramic owl dataset are described in real_ceramic_owl.yaml
Tensorboard logs, checkpoints, arguments and images are saved in the corresponding experiment folder in logs/
.
Using the saved arguments from config.yaml
and the saved checkpoint such as latest.pt
in logs/
, novel views can be rendered using the following command.
python -m tools.render_view
--config logs/our_ceramic_owl_v2/config.yaml
--load_pt logs/our_ceramic_owl_v2/ckpts/latest.pt
Refer to this documentation in neurecon
repo for possible camera trajectories. By default first three views used for training are rendered.
Outputs are saved in the corresponding experiment folder in out/
. By default,the outputs include surface normal, diffuse radiance, specular radiance and combined radiance for each view along with the estimated roughness.
This repository adapts code or draws inspiration from
- https://github.com/ventusff/neurecon
- https://github.com/yenchenlin/nerf-pytorch
- https://github.com/Fyusion/LLFF
- https://github.com/elerac/polanalyser
- https://github.com/sxyu/svox2
@article{dave2022pandora,
title={PANDORA: Polarization-Aided Neural Decomposition Of Radiance},
author={Dave, Akshat and Zhao, Yongyi and Veeraraghavan, Ashok},
journal={arXiv preprint arXiv:2203.13458},
year={2022}
}