Is there any way to use datasets from instant ngp?

[jpbalarini]

Hi, I am trying to train nvfdiffrec straigth from instant-ngp NERF datasets but I'm blocked. The instant-ngp dataset uses images without alpha and it seems like nvdiffrec needs images with alpha. Also, nvdiffrec uses a train/test/val dataset, is there a way to use the same input as instant-ngp uses?

Thanks!

[jmunkberg]

Hello @jpbalarini

Yes, we do require alpha. The foreground segmentation mask is essential for our shape optimization to work. See the paper for details.

We should be compatible with the standard synthetic nerf datasets from https://github.com/bmild/nerf, see e.g., https://github.com/NVlabs/nvdiffrec/blob/main/configs/nerf_chair.json for an example config.

What are the differences between the instant-ngp and the nerf synthetic dataset versions? Our nerf dataset reader is here: https://github.com/NVlabs/nvdiffrec/blob/main/dataset/dataset_nerf.py , hopefully it is easy to customize to your needs.