Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ekf_localizer): add imu option for roll pitch and height estimation #7875

Conversation

meliketanrikulu
Copy link
Contributor

@meliketanrikulu meliketanrikulu commented Jul 5, 2024

Description

We can see that there is a wobbling of ego vehicle points as the vehicle drives through speed bumps.I repeated the tests in the simulation environment and obtained similar results. When I tried to optimize parameters in its current EKF , I saw that I could not achieve good results in its current packet. You can find the tests I have previously done to solve this problem in the issue comments.
Currently, 1D Filter is used for roll, pitch and position z(height) estimation in the ekf_localizer package. The angular velocity data that comes with the twist message and the orientation data in the imu data are not used in the calculations. It is calculated only by reference to the NDT poses that come as input.
However, IMU provides us with orientation information at a very high frequency. By using this data , we can localize roll, pitch and position z more accurately.
This PR suggests adding an option for those who want to use imu data instead of 1D Filters

Related links

Related Issue: #6993

  • Link

How was this PR tested?

  1. Simulation Test:
    In the tests carried out in the simulation environment, you can see below that using imu directly solves the roll, pitch, and position z in EKF instead of calculating them with the 1D Filter:
    sim-COMPARISON-Page-2 drawio
    Buna ek olarak aşağıdaki videoları inceleyebilirsiniz:
    Default Autoware test video is here
    EKF with IMU Integration test video is here
  2. Real World Test:
    Although the same level of performance cannot be achieved in real-world tests due to different factors, there is an improvement. It also seems that the results are further improved by using the 3D Distortion Corrector, which has been newly merged into Autoware.You can find the results below:
    COMPARISON_w-ndt_only drawio
    Real World Test Video is here

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

@github-actions github-actions bot added the component:localization Vehicle's position determination in its environment. (auto-assigned) label Jul 5, 2024
@meliketanrikulu meliketanrikulu self-assigned this Jul 5, 2024
Copy link

github-actions bot commented Jul 5, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@meliketanrikulu meliketanrikulu marked this pull request as draft July 5, 2024 09:33
@meliketanrikulu meliketanrikulu force-pushed the fix(ekf_localizer)add_imu_option_for_roll_pitch_and_height_estimation branch 4 times, most recently from aafa2ba to c2100ed Compare July 5, 2024 10:05
Signed-off-by: Melike Tanrıkulu <melike@leodrive.ai>
@meliketanrikulu meliketanrikulu force-pushed the fix(ekf_localizer)add_imu_option_for_roll_pitch_and_height_estimation branch from c2100ed to bffaf15 Compare July 5, 2024 13:36
@meliketanrikulu
Copy link
Contributor Author

I tested branch with hardware sync data and I get better results.Below, you can review the images captured at the moments when the difference between the instantaneous point cloud and the map was maximized while passing through the speed bump. (Normally instant pc and map points are matching but they move away when passing through a speed bump. You can see here the maximum distances (errors)):

COMPARISON-Page-1 drawio

Also you can see final results in this video
According to these results, we can see that by using hardware sync data, the results are very close to NDT, where we get the best results.

Simulation Tests:
In addition, I tested the simulation tests with VLS128 to see the distortions more clearly. You can see the results in this video.

@meliketanrikulu
Copy link
Contributor Author

Question :

The imu_corrector package does not fill orientation field of imu topic (/sensing/imu/imu_data) in autoware. It just fill angular_velocity and linear_acceleration parts of sensor_msgs/msg/Imu data. Orientation field is comes with default values. You can look here for check that.
I tested this PR with our vehicle's imu sensor output topic . But all of the autoware uses imu_corrector output topic(/sensing/imu/imu_data). I think I need to create PR for adding orientation field to imu_corrector package.
What is your suggestion in here?

@KYabuuchi
Copy link
Contributor

@meliketanrikulu Yes, imu_corrector does not copy orientation from the original IMU data.
The reason for this should be that there is currently no node that uses IMU's orientation.
I think it is good to add the feature to copy orientation to imu_corrector.

@SakodaShintaro
Copy link
Contributor

I'm not sure if copying orientation in imu_corrector is a good idea. I think it's fine to have code to copy, but since it's not used in many cases, I feel like it's increasing unnecessary processing.

Another issue is that according to the documentation, inputting the IMU messages directly into the EKF is different from the Recommended Architecture.
https://autowarefoundation.github.io/autoware-documentation/main/design/autoware-architecture/localization/#recommended-architecture

I think one of the following is needed.
(1) Start a discussion to change the recommended architecture
(2) Implement a new EKF package which receives IMU messages
(3) Implement estimators that output Twist and Pose from the IMU

My personal opinion is that I have nothing against the EKF receiving IMU messages, but would it be possible to achieve this improvement using angular velocity instead of orientation?

@meliketanrikulu
Copy link
Contributor Author

meliketanrikulu commented Jul 11, 2024

I'm not sure if copying orientation in imu_corrector is a good idea. I think it's fine to have code to copy, but since it's not used in many cases, I feel like it's increasing unnecessary processing.

Another issue is that according to the documentation, inputting the IMU messages directly into the EKF is different from the Recommended Architecture. https://autowarefoundation.github.io/autoware-documentation/main/design/autoware-architecture/localization/#recommended-architecture

I think one of the following is needed. (1) Start a discussion to change the recommended architecture (2) Implement a new EKF package which receives IMU messages (3) Implement estimators that output Twist and Pose from the IMU

My personal opinion is that I have nothing against the EKF receiving IMU messages, but would it be possible to achieve this improvement using angular velocity instead of orientation?

Hello @SakodaShintaro @KYabuuchi . Thanks for your comments.
I created discussion --.> https://github.com/orgs/autowarefoundation/discussions/4973 . I would be very happy if you please share your comments.

@SakodaShintaro What is your suggestion for calculating orientation using the angular velocity of the vehicle?
I tested it with dead reckoning. However, this method problematically causes its errors to increase and breaks down after a while.

@SakodaShintaro
Copy link
Contributor

@meliketanrikulu
I haven't read the details of the changes in this pull request yet, so I apologize in advance if I say anything wrong.

There are several papers that integrate IMU with EKF, and since they use acceleration and angular velocity rather than orientation, I think it should be possible to do the same here.

The following paper is one that I recently learned about.

https://arxiv.org/abs/2407.02786

@meliketanrikulu
Copy link
Contributor Author

I created new PR for solving same problem without changing autoware software architecture. #8095
I will close this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:localization Vehicle's position determination in its environment. (auto-assigned)
Projects
Status: Cancelled
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants