How to integrate Autoware without using CAMERA(Problem with perception component)

[KTKTKT0401]

I would like to ask question about integrating autoware without using CAMERA.
Now I'm facing a problem, that my vehicle can't move on to autonomous driving state, because of the perception component error.

I guess this error occurs because I'm not using CAMERA with my vehicle.
Is the process I've done below, is enough for integrating my sensor model for Autoware without the use of CAMERA?

1)Succeeded the connection of lidar, imu, gnss with autoware. I've also connected the vehicle interfaces, such as steering angle, velocity, etc.

2)In 'sensing.launch.xml', deleted the group of launching CAMERA to prevent from launching 'camera.launch.xml'.

3)I've set 'perception_mode' to 'lidar_only', because my vehicle doesn't have a CAMERA.

4)I've set 'use_traffic_light_recognition' to 'false', because my vehicle doesn't have a CAMERA.

P.S.
In my environment, I'm able to succeed the simulation below. So, I think my nvidia drivers and ROS2 environment are installed successfully.
1)Planning simulation using 'sample_vehicle' and 'my original vehicle system'.
2)ROSBAG simulation using 'sample_vehicle'.
3)AWSIM simulation using 'sample_vehicle'.

[VRichardJP]

Autoware should work perfectly without any camera.
What kind of error are you facing?

[KTKTKT0401]

@VRichardJP

Thank you for your reply!
If autoware works well without the usage of camera, maybe the problem occuring in my autoware has a different source of error...

When setting my initial vehicle position and the goal pose, the localization and the route planning seems to be successfull.
After that step, errors below is occuring.

1)when I run 'ros2 run default_ad_api guide.py', the error 'The topic rate error is detected. Please check [perception] components'.

2)I also checked my component state monitor '/system/component_state_monitor/component/autonomous/perception', and it was FALSE(other component state monitor is TRUE).

3)ERROR below occurs. But I could successfully check all the ros2 topic from lidar, imu, gnss, vehicle status, etc., which I integrated using the autoware documentation.

[component_container_mt-7] [ERROR 1720588238.142113468] [system.system_monitor.net_monitor]: Failed to connect socket. No such file or directory (connect_service() at /home/takagi/autoware/src/universe/autoware.universe/system/system_monitor/src/net_monitor/net_monitor.cpp:658)

It would be very helpful if you could give advise to us regarding this issue.

[VRichardJP]

I never used the guide.py, but it looks like its is a very limited tool to diagnose issues.

There are 2 sources of information I suggest you to check for errors/warnings:

• Autoware logs (output of ros2 launch autoware_launch autoware.launch.xml command)
• Autoware diagnostics, which can be visualized with RQT

Note that both will contains a lot of false positive and irrelevent information regarding your issue. For example:

• system.system_monitor.* errors are produced by the hardware monitoring modules and can be ignored. Some of these modules require some specific setup (e.g. to read CPU/RAM/GPU usage) and will complain otherwise.
• Autoware uses topic watchers to monitor and detect missing topics. For example, if you have no camera but a topic watcher is set to monitor some camera related topic, then you may see a few diagnostics stating image data is missing.

Additionally, a single issue can cause many errors to be reported. For example:

• if the vehicle odometer data is missing, a topic watcher may report an error
• since the vehicle odometer data is missing, localization initialization will fail and the localization will report an error
• since the vehicle is not localized, the planning module will fail and may report an error
• etc.

So it is not always easy to find the actual root cause of a reported error.

Anyway, if you can share some logs or screenshots, I may be able to help you :)

[KTKTKT0401]

Hi!

1)The results of 'ros2 topic echo of operation_mode and component_state_monitor'.

2)The results of 'ros2 topic echo /diagnostics'
It looks like my system is trying to receive '/perception/traffic_light_recognition/traffic_signals', and the timeout error is occuring.
I'm now checking the autoware launch file, to see if I don't launch any components regarding the camera.

3)Autoware logs (output of ros2 launch autoware_launch autoware.launch.xml command)
Same as the 2). Trying to read '/perception/traffic_light_recognition/traffic_signals'.

[VRichardJP]

As you can see, you have a topic_state_monitor which complains something is wrong with the traffic light pipeline. You have no camera so this is totally normal and you can ignore the error.

I think you should be more concerned by the localization errors. I see you have:

• NDT align server failed = localization initialization failed for some reason
• The mahalanobis distance xxx is over the limit = EKF localizer received some localization data that does not match the current known state. For example, vehicle velocity report suddenly jumps to 30km/h but EKF was pretty confident the current velocity was 15km/h.

From these few points, I guess you could not set the initial position of the vehicle? What does Rviz view looks like?

[KTKTKT0401]

From these few points, I guess you could not set the initial position of the vehicle? What does Rviz view looks like?

I could set the vehicle's initial pose, and I coud also set the goal pose. The route is also calculated.
I also can see the conatenated pointcloud, and it is published in 10Hz. And on rviz2, I can see my classification works as well(I've checked the pedestrian, car, truck label and its current velocity, etc.).

My vehicle status, such as velocity and steering angle is also displayed collectly on rviz2 as well.

Should I try running the vehicle manually, and check whether the localization success while the vehicle moving?

[VRichardJP]

It is difficult to tell without more information.

If you want to get rid of the traffic light related error, you could edit src/launcher/autoware_launch/autoware_launch/config/system/component_state_monitor/topics.yaml and remove the following lines:

https://github.com/autowarefoundation/autoware_launch/blob/c20e08c2ca2cb3edfd3350b2ad52b09bbf310a2e/autoware_launch/config/system/component_state_monitor/topics.yaml#L79-L90

But I don't think it will resolve your issue. AFAIK, these topic watchers have no effect because use_emergency_handling in vehicle_cmd_gate is set to false by default. Most likely the issue is reported somewhere in the logs or in the diagnostics (you can visualize diagnostics easily with RQT)