run.sh

[catherine-qian]

For the run.sh script:
CUDA_VISIBLE_DEVICES=2 python train.py
--validation_on
--dataset mp3d
--img_path /data1/kranti/audio-visual-depth/dataset/visual_echoes/images/mp3d_split_wise
--metadatapath /data1/kranti/audio-visual-depth/dataset/visual_echoes/metadata/mp3d
--audio_path /data1/kranti/audio-visual-depth/dataset/visual_echoes/echoes/mp3d/echoes_navigable
--checkpoints_dir /data1/kranti/audio-visual-depth/checkpoints
--init_material_weight ./checkpoints/material_pre_trained_minc.pth

Could you please let me know how to get the images/ and the metadata/?

[krantiparida]

The metadata is available inside the dataset folder of this repo. For the images, we cannot share the extracted frames directly due to the restriction from Matterport. However, you can use the steps mentioned in the SoundSpaces repo to generate those frames.

[catherine-qian]

thanks for the reply.

Could you please kindly share the scripts to generate those image frames？
I have checked soundspaces but cannot successfully generate them.

Best

[SaschaHornauer]

It's been a while but could I ask for the same script? I checked the SoundSpaces repo but it only shows how to create one pkl file for each scene but not how to create your train.pkl file which the code seems to search?