Big PCD Maps and AW Performance Problems

[masierralomby]

Good afternoon!

I am currently working with really big Point Cloud Maps. After getting the Raw Data, I am processing it to get a Dense PCD. After that, I will remove outliers, roads and then apply a voxel grid to down-sample to 0.2 m,

According to Autoware Documentation, that's the limit to not compromise Localization performance.

Even with all of this, I am ending with a .PCD file of around 6.5 GB. I am also using partial and differential loading functions of the Map Loader. I am using divided point cloud data too.

With that size, the PCD is not properly being loaded and I am not even able to visualize it in RViz. I am not able to use my vehicle as the Map Component will judge as a failure and the status for that component will be false.

Has anyone experienced something similar and has gotten to properly work with super big point cloud maps?

My current environment is:
Ubuntu 22.04
32.0 GiB RAM
32.0 GB Swap
256.1 GB Disk
12th Gen Intel® Core™ i7-12700TE × 20
NVIDIA RTX 2000 Ada Generation/PCIe/SSE2

I am working Autoware Release for January 2024.

If anyone could have some light on this situation it would be really appreciated.

Thank you in advance.

[YamatoAndo]

@masierralomby Hi.
The size is too large and will likely exceed the topic communication limit.
As it is, it won't be possible to display in RViz, so please use a downsampled topic(/map/output/debug/downsampled_pointcloud_map
).
To enable downsampling, you need to set enable_downsampled_whole_load to true.

[liuXinGangChina]

hi @masierralomby ，just like Ando-san said you can swith that param on in the config file to significantly reduce the transport between Maploader and rviz

I also suggest you to try pcd divider to cut the large map into pieces， AW will upload related pcd piece when it is needed。Is that right ？ Ando-san。

happy hacking！

[masierralomby]

@YamatoAndo @liuXinGangChina
Thank you so much for the answer! I was already trying divided pcd. I am going to try the downsampled topic.
As a big solution I am going to work with smaller maps and then restart as needed.