CARLA2NMR Viewer

CARLA2NMR Viewer is an application designed to support the visualization of data in COLMAP format, including LiDAR point clouds and camera poses. This viewer application also supports cropped LiDAR point clouds (based on camera POV), Gaussian SLAM (in progress), and LiDAR odometry using KISS-ICP.

Features

• COLMAP Format Support: Load and visualize data in COLMAP format.
• LiDAR Point Cloud Visualization: Display full LiDAR point clouds as well as cropped point clouds based on camera POV.
• Camera Pose Visualization: Show camera poses from the loaded data.
• KISS-ICP LiDAR Odometry: LiDAR odometry using kiss-icp.
• Gaussian Splatting Training: Gaussian Splatting Training with gsplat backend.
  • Scale Regularization: Preventing long, spikey gaussians by proposing a scale regularizer that encourages gaussians to be more evenly shaped. For more details, refer to splatfacto-nerfstudio and PhysGaussian.
  • Visualization of Training 3DGS: Support rendering and training 3DGS in web-based viewer. For more details, refer to nerfview.

Usage

This project requires Python 3.8+. To run the application, follow the steps below:

1. Clone the repository:

git clone https://github.com/zhumorui/CARLA2NMR.git

2. Navigate to the repository directory:

cd CARLA2NMR

3. Installation:

pip install -r requirements.txt

4. Install kiss-icp Python API:

git clone git@github.com:zhumorui/kiss-icp.git
cd kiss-icp
make editable 

5. Run the app:

python src/main.py

Workflow

graph TD
    A[Start] --> B
    B --> C
    C --> D
    D1 --> D2
    D2 --> D3
    D --> E
    E & C --> G
    G1 --> |Gaussian Splatting Registration|G2
    G --> J[End]

    subgraph B[CARLA Simulator]
    end

    subgraph C[CARLA Dataset]
      C1[Images]
      C2[Lidar Point Cloud]
      C3[Camera Parameters]
    end

    subgraph D[Lidar Preprocessing]
      D1["Coordinate transformation (Optional)"]
      D2[Cropping Based on POV]
      D3[Color Mapping]
    end

    subgraph E[Lidar Odometry]
      
    end

    subgraph G[Gaussian Splatting]
      G1[Training]
      G2[Combine Multiple Trajectories]
    end
    

Loading

Demo

1. Visualize images and camera poses.

2. Lidar odometry with kiss-icp method.

3. Gaussian Splatting training with gsplat backend.

Future Work

• Gaussian SLAM: support Gaussian SLAM.

Acknowledgements

I thank the authors of the following repositories for their contributions to this project:

• viser
• gsplat
• colmap
• kiss-icp
• nerfview
• Gaussian-SLAM