Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIXED] Bug in the self-attention GNN. #15

Closed
zgojcic opened this issue Jun 2, 2021 · 5 comments · Fixed by #14
Closed

[FIXED] Bug in the self-attention GNN. #15

zgojcic opened this issue Jun 2, 2021 · 5 comments · Fixed by #14
Labels
bug Something isn't working

Comments

@zgojcic
Copy link
Collaborator

zgojcic commented Jun 2, 2021

As correctly noticed by @zhulf0804 (thank you), there was a bug in our implementation of the GNN, which actually led to lower performance of PREDATOR. The bug was fixed in the Pull Request #14. We have now retrained the model on the 3DMatch dataset and obtained a higher performance (see Figure below). We are also retraining PREDATOR on other dataset and will update the tables in the following days.

Note, due to the change of the GNN architecture old pretrained models will not work anymore, so make sure to always have the latest version.

image_2021_06_02T09_21_10_068Z

@zgojcic zgojcic linked a pull request Jun 2, 2021 that will close this issue
@zgojcic zgojcic added the bug Something isn't working label Jun 2, 2021
@ShengyuH ShengyuH pinned this issue Jun 3, 2021
@zgojcic
Copy link
Collaborator Author

zgojcic commented Jun 10, 2021

Old the 3DMatch models were now retrained after fixing the bug:
The performance of PREDATOR improves up to 8 percent points on the 3DLowMatch:
3dmatch_result

The bug fix also reflects itself in the ablation studies, where the benefit of the OA module and conditioning the features are now even more clear:

ablation

sampling

@zgojcic
Copy link
Collaborator Author

zgojcic commented Jun 10, 2021

In the similar manner we are now also retraining the ModelNet and KITTI models and will update the arxiv paper once all results are available. If you have any questions in the meantime please just contact us directly.

Best,
Zan

@ShengyuH
Copy link
Member

Updates on Modelnet and KITTI:

image
image

arxiv paper and pre-trained weights will be updated in following days.

Best,
Shengyu

@ShengyuH
Copy link
Member

ShengyuH commented Aug 1, 2021

Updates on MinkowskiEngine-based predator:

image

Pretrained weights are updated now, paper will be updated in the following days.

wget --no-check-certificate --show-progress https://share.phys.ethz.ch/~gsg/Predator/weights.zip

best,
Shengyu

@zgojcic
Copy link
Collaborator Author

zgojcic commented Aug 9, 2021

This issue has now been fixed for all the models with both backbones (MinkowskiEngine and KPConv). The fix increases the performance across all tasks. New values can be seen in the updated version of the paper.

We have also updated all the pretrained models such that it should be possible to reproduce the latest values.

Thank you for your patience.

Best,
the authors

@zgojcic zgojcic closed this as completed Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants