6D-NDT(X-Y-Z-R-G-B)

[mertyavuz41]

Hi there,

I know 3D-NDT already very robust and powerful but 3D-NDT tend to be failure in such a featureless environments because of its highly rely on geometrical features.

Increasing dimension of NDT might help to handle such a featureless environments, like 6D NDT(xyzrgb). In this way, the road line will be considerable.

Here the comparison with 3D-NDT and 6D-NDT;

https://www.diva-portal.org/smash/get/diva2:276162/FULLTEXT02.pdf

In this paper include a studies that explain and compare colored NDT at page 125.

the conculusion of 6D-NDT:

In this chapter, a kernel-based extension to the NDT scan registration algorithm
has been presented. Colour-NDT is more robust than purely geometric NDT
for 3D scans with little geometric features.

...

Colour-NDT can successfully be applied in such cases. As a general recommendation for colour-aware 3D
registration, the combined registration algorithm of Huhle et al. — but using
Colour-NDT instead of 3D-NDT — should be used.

I couldn't find any studies that shows how the execution time complexity will be, but obviously 6D-NDT will run more slowly.

[mertyavuz41]

Here the dataset for testing of colour-aware registation studies:

https://netorgft4229778-my.sharepoint.com/:u:/g/personal/mert_adastec_com/EdjtsJHaM1pFpUrtxri3e74BsTx7FhPsQeF2W3OP49Z-cg?e=0jc5Eg

I also prepared ground colored point clouds to see how the colour-awared registration will behave with in featureless cloud.

folder structure:

| - keyframes/
| | - seq1/
| | | - dash_center/
| | | | - dash_center.jpg
| | | | - info.txt
| | | - dash_left/
| | | | - dash_left.jpg
| | | | - info.txt
| | | - dash_right/
| | | | - dash_right.jpg
| | | | - info.txt
| | | - lidar/
| | | | - cloud.pcd
| | | | - cloud_rgb.pcd
| | | | - ground_cloud.pcd
| | | | - ground_cloud_rgb.pcd
| | | | - info.txt
| | - seq2/
...

[VRichardJP]

I have not read the paper so excuse my ignorance, but I am unsure how color could help to tackle NDT issues. NDT is the weakest in environments lacking geometric features: tunnels with smooth walls, country-side road with nothing around but grass, etc. But these environments also typically lack color features: tunnel walls are grey, country-side road is mostly grey, the grass around is green. I don't say adding color would not improve accuracy, but I am not sure it would really help much in these kind of edge cases.

Also, I am curious how to generate RGB pointcloud. Is there any LIDAR sensor that can output RGB? or we need to use extra cameras to paint the pointcloud? If extra processing is required, is the accuracy gain on NDT side worth the extra latency?

[mertyavuz41]

Hi @VRichardJP,

First of all, thanks for your nice questions.

1-) Yes, you're right. The NDT highly depends on geometrical feature around, the problem that we want to solve stars here. The main idea here is the roads commonly have lines, if we can make NDT considerable to these road lines we can use it. How we can make NDT color awared? The standard NDT is calculating probability of each point according to target distributions then NDT score is average of this calculation, then optimization and line search process continue to find best solution. In some way if we can add a weight according to color differences between correspondences for score calculation, we make color awared NDT(But this is still a approach so I'm not sure how it will work).

2-) I believe this approach will solve the tunnel case that the roads in the tunnel has road markers(or any colored texture in it), but if environment is unique as color and geometrically, it will not have a solution for these cases.

3-) I colorized point cloud with externals cameras, you can find dataset link above comment.

4-) I couldn't find benchmark values for execution time of color awared NDT so the only way to learn this is try. According to my experiences, the main reason of exceed of execution time in standard NDT is extra iteration number of optimization and line search due to bad initial pose.
My first idea is the kd-tree can be still stand 3D(nearest neighboring search), we will add extra process to score calculation. Moreover, It may be better than standard NDT beacuse, the line search will find the correct direction for best solution. We have to work, and see the results :)

I want to the remind, this is still a approach so we can see the useless results end of the work but the paper conclusion says it will be more robust, and I'm open to any idea.

Thanks,