Add Visual SLAM method in mapping and localization pipeline

[StepTurtle]

Hi everyone,

We are work on visual slam methods in mapping work group and we want the implement to Autoware.

There is a nice work in this link:

• https://github.com/TUMFTM/ORB_SLAM3_RGBL

This work combine ORB features and lidar point cloud as a depth map and it works like RGB-Depth cameras. It gives us some advantages:

• It provides more accurate odometry than classical ORB methods.
• It works faster, so it will be use for high-speed mapping and localization.

I would like to know if you have any ideas and comments about this method. Thanks.

[shmpwk]

cc. @KYabuuchi

[StepTurtle]

UPDATE: I want to explain our purpose and road map a little bit more.

With this work, we try to create alternative solution for laser scan mapping and localization (eg. NDT). Our main purpose is create a geo-referenced feature map with ORB_SLAM3. We will calibrate lidar, camera and GNSS/INS sensors and we obtain camera poses from GNSS/INS sensor. When we create feature map with ORB_SLAM3, we use GNSS/INS poses for camera poses. In this way, we avoid driftting and deviating from road. So, if we want to localize on map later, we will use the georeferenced feature map. We think that with this method, we could get better odometry.

I would like to tell you our road map step by step so that it is clearer:

1) Firstly, we try to figure out how can we feed ORB_SLAM3 with GNSS/INS poses. For this step I will use KITTI dataset and KITTI ground truth. I will use ground truth poses like GNSS/INS poses. When we do this step correctly, we will use GNSS/INS.

2) I will calculate transforme between lidar-GNSS/INS and lidar-camera.

3) I will feed ORB_SLAM3 with GNSS/INS poses as I did with KITTI dataset, and I will create Geo-referenced feature map

4) I will localize on both feature and laser scan map and I will evaluate the scores.

5) If everyone like this work and it works well, we try to implament to Autoware.

I'm waiting for your questions if there are parts that is not clear.

[KYabuuchi]

Hi! Thanks for sharing your interesting idea.
I have used ORB_SLAM with modifications in the past, too.

I have some concerns with your plan.

(1) performance
ORB_SLAM3_RGBL is said to be superior to ORB_SLAM3 (stereo mode). However, I wonder if ORB_SLAM3_RGBL is superior to other methods that uses LiDAR.
In KITTI's benchmark ranking, LOAM which uses only LiDAR is much superior to ORB_SLAM2 (stereo mode).

(2) license
ORB_SLAM3 is GPL3, while Autoware's license is Apache-2.0. If you plan to incorporate it into Autoware in the future, please be careful about license contamination.

(3) planning/control
Autoware requires Lanelet2 map for planning and control. If your goal is to run Autoware with a visual feature map, you need to consider linking Lanelet2 and feature map.

I hope it will go well :)

[StepTurtle]

Thank you very much for your valuable feedback.

For your first concern;
ORB_SLAM3_RGBL is behind a little bit in terms of translation and rotation errors but it manages to get ahead in terms of speed especially at the start of localization. So we aim to reduce these errors with the geo-referenced (ground truth) feature map we aim to create.

About your second concern, I don't know much about licenses but I'll heed your warning.

For your third concern;
We are considering solutions to link it with Lanelet2, if we get the results we want, we will work on the next stage. I also aim to create point cloud map besides the geo-referenced feature map. I think we can use this point cloud map for creating HD map and we can use ORB_SLAM3_RGBL results with Lanelet2.

Thanks you again for your feedback. ^_^