Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Real-Time Rendering support using SIBR. #9

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

rahul-goel
Copy link

Hi.

I wanted to visualize these gaussians in real-time so I added the support for it. The following video shows a quick test I did on the synthetic ficus scene which went till 3000 iterations only.

Screencast.from.04-12-23.02.03.04.AM.IST.webm

Firstly, I've added another method to save the plyfile which is saved as point_cloud_sibr.ply. I apply the change to scaling and opacity and then save the values. Accordingly, I don't pass these values through the activation function in the real-time viewer. The real-time viewer will look for the point_cloud_sibr.ply file.

The Cuda Rasterizer files have been taken from your rasterizer implementation and modified according to the arguments that the real-time viewer passes. To the kernel_size parameter, a value of 0.2 is passed.

The instructions to install and run the SIBR viewer are the same as the one mentioned in the original gaussian splatting repository: https://github.com/graphdeco-inria/gaussian-splatting. Their viewer downloaded the cuda rasterization submodule. Instead, I've just put it directly in the directory. The instructions to launch the viewer are also the same.

I've tested this only on an Ubuntu Machine. I cannot test on Windows.

@niujinshuchong
Copy link
Member

@rahul-goel Thank you so much for the PR. I think it would be better to create a fused ply that is compatible to the original 3DGS since in this case we just need to update the rasterisation. I just push the code for creating fused ply file which also consider activation function.

Btw, I think it would be good to have a script of this conversion since people can use it only when they need it otherwise it takes a lot storage if we save it always after training.

The kernel_size is 0.1 by default.

Would be great if you can update it accordingly to the latest commit and I will merge it then. Thank you very much again for the PR.

@rahul-goel
Copy link
Author

Hi.

Cool idea to invert the activation functions. I've made changes to the SIBR accordingly. The kernel size is changed to 0.1.

To visualize, the filename should still be "point_cloud_sibr.ply" and be in the same directory as "point_cloud.ply". That's because SIBR expects this directory structure.

@niujinshuchong
Copy link
Member

@rahul-goel Thanks for the update! Can you make it optional for saving the SIBR point cloud and default it to false?

@rahul-goel
Copy link
Author

I don't understand what you mean by that.

@niujinshuchong
Copy link
Member

Hi, I mean now the SIBR points are always saved along with the original point cloud and this doubles the storage. I think it would be better to have a flag that controls whether we want to save it or not.

@rahul-goel
Copy link
Author

Ah. I missed removing that. Fixed.

@niujinshuchong
Copy link
Member

Hi, thanks for the update. Since now only the SIBR project is update and there will be 2000+ files in this commit, I think it would be better to make it as an external module as done in the original 3DGS project. So could you make it as a standalone module in your own fork and i will link it and update the readme accordingly.

@rahul-goel
Copy link
Author

Hi.

I also thought so. But I think it is better if you setup the module and host it yourself instead of me.
Because if there is a change in your rasterizer, then I would have to make that change in the submodule also.

So I think it is best if you do it. Once done, you can close this PR without merging.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants