SLAM with deep learning feature and descriptors (SuperPoint)
This work includes changing the ORB-SLAM2 pipeline in order to work with deep learning descriptors like SuperPoint. Eventually, we would like to use SuperGlue to improve matching even further in this pipeline
Docker Container: superslam
docker pull cklammer/superslam
Code for Superpoint v2 (Superpoint-opt) which is vanilla SuperPoint with a MobileNet backbone and support for both RGB and Grayscale inputs: Superpoint-opt
Superpoint v1 Vocabulary Superpoint v2 Vocabulary
- Docker container to work with deep learning descriptors and setting environment
- Support with DBoW3 and C++17 thanks to GCN_v2's repo
- Fixing bugs in code with regard to loop closures and tracking from the community
- Improve QoL when using standard library
- Training a vocabulary for Superpoint and a MobileNet variant Superpoint_v2
- Reworking matching to work with deep learning descriptors, adding nearest neighbor matching from GCN_v2's repo
- Easier data pipelining with datasets like TartanAir and TUM
Look to see additional improvements coming soon!
Notice: As I'm finishing up my master's this December I will be a bit busy, I likely will not revisit this until around December unless I decide that I can continue this work in a course project. This is a research topic that I am still very interested in but don't have the time right now. If you are interested in collaborating and contributing to this, please send me an email!
| Status | Target Completion | Contribution |
|---|---|---|
| ☑ | End May | Include trained vocabularies used for Superpoint_v1 and Superpoint_v2 and shell scripts used to pull them down |
| In Progress.... | End May-Begin June | Investigate performance issues for Superpoint v1 |
| ☐ | ? | Validate all code is migrated and switch over docker image to have the latest pre-loaded |
| ☐ | ? | Integrating our Superpoint_v2 (Superpoint-opt) into this repo |
| ☐ | ? | Adding a switch allowing use of both ORB and Superpoint frontends |
| ☐ | ? | Speed up and better acceleration for GPUs |
| ☐ | ? | Adding SuperGlue/GCN/COTR for matching |
| ☐ | ? | Begin summarizing and finalizing work and findings for a workshop/conference |