Custom dataset

[samueleruffino99]

Hello, I would like to run your model on a custom dataset.
I am not quite sure besides the 6 RGB images and can bus could affect the result of your algo.
Do you think that just the domain shift in the video could impact the result that bad?
I would simply like to run it on multi-view images (I have all transformations and imu data).
Thank you!!

[cnzzx]

Thanks for your interest. It's hard to say how well the model will work in the zero-shot scene, and we didn't test that. Maybe it's better to train on your custom dataset?

[samueleruffino99]

Unfortunately is a private custom dataset with no annotation, so it might be quite difficult to get these annotations.
I wanted to double-check if 6-views images and can_bus data (inputs of the model) can affect the results.
Or do you think also other inputs might influence the results of the algorithm?
Moreover, maybe you are using some transformation/projection matrices and I haven't constructed them correctly.
Thank you very much!!

[samueleruffino99]

Or this issue could be caused by images only? Because i have seen that lidar2img transformation matrix are used to construct some anchor points. But I am using the same lidar2img matrices in another algorithm and apparently those are correct.
So I really do not know which could be the problem.

[cnzzx]

Are you meeting some problems when deploying the pretrained weights? There will be many factors to affect the performance. For example, 1) how is the image data normalized and preprocessed? 2) is the can_bus-like information aligned well with our processing (eg. the same units)? 3) is the distribution shift of your custom dataset from nuScenes severe? etc.

[samueleruffino99]

I have tested your algo with can_bus data all equal to zeros and it works the same (I am using the simple version of the model). Moreover i am using the same preprocessing technique of yours, I didn't change anything.
One thing is that the prediction I get are upside down in the vertical axis of BEV, compared to the one I get with nuscenes, I am not sure where it could come from actually, since the images are in the correct order.
Nuscenes lidar coordinate system, if i am not worng is: x: right, y: forward, z: up. While my coordinate system is x: forward, y: right and z: up.
Maybe how could I check whether I am using the same coordinate system as yours?
i would really appreciate some suggestion :)
Thnak you very much!! :)