Proposal to integrate Eagleye, a GNSS/IMU-based localizer, into Autoware

[rsasaki0109]

@yukkysaito @mitsudome-r @xmfcx (@n-patiphon)
On behalf of MAP IV. Inc, I am pleased to announce a proposal to integrate Eagleye, our open-source GNSS/IMU-based localizer, into the localization stack of Autoware. Eagleye uses low-cost GNSS and IMU sensors to provide vehicle positioning and orientation, offering a cost-effective alternative to LiDAR and point cloud-based localization.

Although the current version of Eagleye requires wheel speed information, there are plans to remove this requirement in the future, making Eagleye even more accessible to Autoware users. With this planned integration, Autoware users will have the option to choose between the existing LiDAR and point cloud-based localization stack or the GNSS/IMU-based Eagleye localizer, depending on their specific needs and operating environment.

In our planned integration, there are two ways to utilize Eagleye’s result in the Autoware localization stack:

1. Feed both Twist and Pose from Eagleye into the EKF localizer.

2. Feed only Twist into the EKF localizer.
   

By using both Twist and Pose from Eagleye, Autoware can localize without using LiDARs. Meanwhile, feeding only Twist into the EKF localizer is expected to improve the stability of NDT scan matching by providing a more accurate guess pose for scan matching.

We have already started working on the proposed integration in Q4 of 2022, and several tasks have already been completed to make Eagleye compatible with Autoware and to allow for testing the integration:

• Modifying Eagleye to support TwistWithCovarianceStamped as its speed input to match Autoware’s specifications
• Adding a fix to handle EKF not initialized without NDT
• Adding a new geoid model to Eagleye
• We have also modified autoware_launch on a private fork of Autoware for testing. This modified autoware_launch allows users to easily enable the use of Eagleye outputs in Autoware.

As for the current progress, we are now in the testing phase. We have just finished the first testing on a real vehicle. We performed the test in an Urban area surrounded by high buildings. The vehicle was driven by a human driver (NOT in an autonomous mode). The goal of this test was to determine if the estimated positions were accurate enough in actual driving scenarios. The results were promising as the estimated positions remained within the lane throughout the test. The following is the video of the test.

https://www.youtube.com/watch?v=_AJmJfGuEoo

We plan to finish implementation and testing by the end of Q1 of 2023. During this time, we plan to have another two tests. One with a human driver and the other in full autonomous driving mode. Moreover, we will transfer changes of autoware_launch to a public fork of Autoware in preparation for opening a PR. We have regular discussions regarding the details and progress of this integration in the Mapping Work Group, but we will also post future updates in this Discussion as well.

Please note that this integration is not meant to replace the current localization stack but to provide Autoware users with another option. We believe that the integration of Eagleye will enhance the functionality and versatility of Autoware.

We invite the Autoware community to stay tuned for updates on the integration process and to provide feedback and suggestions. Your contributions are essential for the continued improvement and development of Autoware.

Thank you for your support, and we look forward to your feedback!

[yukkysaito]

Nice activity 👍 If you have any problems, I will do our best to support you, so please feel free to ask anything.

[rsasaki0109]

I submitted pull requests.
#3261
autowarefoundation/autoware_launch#200
autowarefoundation/autoware.universe#2848

[mitsudome-r]

@rsasaki0109 It would be nice if you can also create PR to Autoware Documentation under "How to Guide" to explain how people can use Eagleye.

[rsasaki0109]

That's important, I would like to create a PR.

Work is in progress on the following branches
https://github.com/MapIV/autoware-documentation/tree/add_eagleye_in_how-to-guides
PR
autowarefoundation/autoware-documentation#334

[rsasaki0109]

PRs merged except docment.

[n-patiphon]

Hi @meliketanrikulu! To follow up on your question last night, I checked with the developer (@rsasaki0109), and it seems like there won't be any changes to the EKF localizer in this integration. However, I've just heard that there are some modifications needed in pose_initializer to keep AD API working properly.

[meliketanrikulu]

Hello @n-patiphon . Thanks for information. Yes, pose_initializer may need to be changed to trigger EKF.

[rsasaki0109]

I have addressed this issue.
autowarefoundation/autoware.universe#2904

[VRichardJP]

Hi!
Sorry to dig out an old discussion. Have you made any comparison between the following setups:

1. eagleye Twist & pose -> EKF
2. eagleye Twist + NDT pose -> EKF
3. gyro odom Twist + NDT pose -> EKF (default autoware)
4. GNSS pose + gyro odom Twist + NDT pose -> EKF (not implemented in autoware EKF, but it is not difficult to add)

In particular, I am curious how 2. and 4. differ, as localization would use the same data but with different algorithms.

Thanks!

[rsasaki0109]

@VRichardJP
For setups 1-3, we have conducted detailed evaluations and plan to share the results eventually (perhaps in Mapping WG). Setup 4 is also currently under evaluation.

For setup 2, compared to setup 3, the performance of Twist is improved, which results in a better localization outcome. This is particularly helpful in environments with fewer distinctive features, where NDT can become unstable. The improved Twist helps reduce the number of iterations needed for NDT.

As for setup 4, while it is indeed possible, it is currently under investigation with an estimated completion timeline around September. For example, issues such as misaligned absolute coordinates in the point cloud map may cause the localization to not work properly. If this combination works well, I believe it will enable more robust localization.

[enescingoz]

@rsasaki0109 is there any updated documentation to run eagleye with autoware?

[rsasaki0109]

Please check the link provided.
https://github.com/autowarefoundation/autoware-documentation/blob/main/docs/how-to-guides/integrating-autoware/launch-autoware/localization/eagleye/index.md

[noyansongur]

@rsasaki0109 Thank you for your response. Just to follow up on @enescingoz's question, we are trying to run Eagleye with one of the provided datasets on Autoware documentation, specifically rosbag replay data. However, we have not been able to localize using Eagleye. Is there a dataset that you confirmed to be working with Autoware and that you can share with us?
Thank you very much in advance!

[rsasaki0109]

This PR's rosbag has been verified to work with Autoware and Eagleye.
autowarefoundation/autoware.universe#6144

[noyansongur]

Thank you @rsasaki0109 for your prompt response. I just downloaded the rosbag and it looks like there is no twist topics in there. All I can see are the following topics:
/clock | Type: rosgraph_msgs/msg/Clock
/sensing/camera/traffic_light/camera_info | Type: sensor_msgs/msg/CameraInfo
/sensing/camera/traffic_light/image_raw | Type: sensor_msgs/msg/Image
/sensing/gnss/pose | Type: geometry_msgs/msg/PoseStampe
/sensing/gnss/pose_with_covariance | Type: geometry_msgs/msg/PoseWithCovarianceStamped
/sensing/gnss/ublox/nav_sat_fix | Type: sensor_msgs/msg/NavSatFix
/sensing/gnss/ublox/navpvt | Type: ublox_msgs/msg/NavPVT
/sensing/imu/tamagawa/imu_raw | Type: sensor_msgs/msg/Imu
/sensing/lidar/top/pointcloud_raw_ex | Type: sensor_msgs/msg/PointCloud2
/vehicle/status/velocity_status

I am curious what would be the following config parameters for this rosbag to run eagley as pose_source and twist_source?
especially:

• twist type and twist topic
• sub-gnss type and topic
• use_gnss_mode

Thank you so much once again!

[rsasaki0109]

The /vehicle/status/velocity_status topic, related to wheel velocity, is converted into the twist topic by a node vehicle_velocity_converter .
The /sensing/gnss/ublox/navpvt contains speed from GNSS and is set as gnss.twist_source.
The gnss.pose_source is /sensing/gnss/ublox/nav_sat_fix.
There's no need to configure the sub-gnss type and topic as it is not present in this rosbag.
The use_gnss_mode doesn't need to be changed from the standard settings.