Why is test_scene_as_chunks running on input instead of target and what is the definition of Valid Pixels and 3D reconstruction loss?

[ferbrjan]

Good afternoon,

After successfully running the testing of SPSG on our data, I was studying the code of test_scene_as_chunks.py and came across a weird part of the code. When running the trained model spsg.pth using the model() function, one of the arguments is chunk_input. From what I understand, this chunk_input variable refers to the incomplete version of the scan, while the testing part should run on the complete scan (I assume that is chunk_target_sdf variable). See screenshot below: (line 138 in test_scene_as_chunks.py

)

To find more information about this, I was rereading your paper to understand the process entirely, and I came across two more things that are somewhat unclear to me. Can you please specify what is meant by valid pixels in section 4.2?

Another unclear thing is the 3D geometric reconstruction loss. Can you please define what is meant by this term?

Thank you very much for your help,

Regards,

Jan

[wtiandong]

Good afternoon,

After successfully running the testing of SPSG on our data, I was studying the code of test_scene_as_chunks.py and came across a weird part of the code. When running the trained model spsg.pth using the model() function, one of the arguments is chunk_input. From what I understand, this chunk_input variable refers to the incomplete version of the scan, while the testing part should run on the complete scan (I assume that is chunk_target_sdf variable). See screenshot below: (line 138 in test_scene_as_chunks.py

)
To find more information about this, I was rereading your paper to understand the process entirely, and I came across two more things that are somewhat unclear to me. Can you please specify what is meant by valid pixels in section 4.2?

Another unclear thing is the 3D geometric reconstruction loss. Can you please define what is meant by this term?

Thank you very much for your help,

Regards,

Jan

Since the authors seldom reply to issues on Github, I would like to try to answer your questions.

First, "valid pixels". When you render a TSDF volume at a specific camera pose, not all pixels on the rendered image are filled. At some locations, there isn't "mesh" (i.e, empty), but the camera "sees" these locations. So the empty locations rendered on the image are invalid pixels. See here

Second, "3D geometric reconstruction loss". This is just loss = |TSDF_pred - TSDF_target|.
See here

[ferbrjan]

Good afternoon,
After successfully running the testing of SPSG on our data, I was studying the code of test_scene_as_chunks.py and came across a weird part of the code. When running the trained model spsg.pth using the model() function, one of the arguments is chunk_input. From what I understand, this chunk_input variable refers to the incomplete version of the scan, while the testing part should run on the complete scan (I assume that is chunk_target_sdf variable). See screenshot below: (line 138 in test_scene_as_chunks.py

)
To find more information about this, I was rereading your paper to understand the process entirely, and I came across two more things that are somewhat unclear to me. Can you please specify what is meant by valid pixels in section 4.2?

Another unclear thing is the 3D geometric reconstruction loss. Can you please define what is meant by this term?

Thank you very much for your help,
Regards,
Jan

Since the authors seldom reply to issues on Github, I would like to try to answer your questions.

First, "valid pixels". When you render a TSDF volume at a specific camera pose, not all pixels on the rendered image are filled. At some locations, there isn't "mesh" (i.e, empty), but the camera "sees" these locations. So the empty locations rendered on the image are invalid pixels. See here

Second, "3D geometric reconstruction loss". This is just loss = |TSDF_pred - TSDF_target|. See here

Perfect! This helps a lot!

Now the only missing part is the usage of chunk_input. Hopefully, someone will clarify that.

[wtiandong]

BTW, may I ask what gan loss you use and how many epochs you train the network when reproducing SPSG code? I notice that training is very slow when using vanilla gan loss.

Good afternoon,
After successfully running the testing of SPSG on our data, I was studying the code of test_scene_as_chunks.py and came across a weird part of the code. When running the trained model spsg.pth using the model() function, one of the arguments is chunk_input. From what I understand, this chunk_input variable refers to the incomplete version of the scan, while the testing part should run on the complete scan (I assume that is chunk_target_sdf variable). See screenshot below: (line 138 in test_scene_as_chunks.py

)
To find more information about this, I was rereading your paper to understand the process entirely, and I came across two more things that are somewhat unclear to me. Can you please specify what is meant by valid pixels in section 4.2?

Another unclear thing is the 3D geometric reconstruction loss. Can you please define what is meant by this term?

Thank you very much for your help,
Regards,
Jan

Since the authors seldom reply to issues on Github, I would like to try to answer your questions.
First, "valid pixels". When you render a TSDF volume at a specific camera pose, not all pixels on the rendered image are filled. At some locations, there isn't "mesh" (i.e, empty), but the camera "sees" these locations. So the empty locations rendered on the image are invalid pixels. See here
Second, "3D geometric reconstruction loss". This is just loss = |TSDF_pred - TSDF_target|. See here

Perfect! This helps a lot!

Now the only missing part is the usage of chunk_input. Hopefully, someone will clarify that.

[ferbrjan]

I used the default setting from the example command. It was slow but the trained model was working faster than the one provided by the authors.

sorry but I have not experimented with the training too much. :-(