Skip to content

Latest commit

 

History

History
5 lines (3 loc) · 1.94 KB

2405.19321.md

File metadata and controls

5 lines (3 loc) · 1.94 KB

DGD: Dynamic 3D Gaussians Distillation

We tackle the task of learning dynamic 3D semantic radiance fields given a single monocular video as input. Our learned semantic radiance field captures per-point semantics as well as color and geometric properties for a dynamic 3D scene, enabling the generation of novel views and their corresponding semantics. This enables the segmentation and tracking of a diverse set of 3D semantic entities, specified using a simple and intuitive interface that includes a user click or a text prompt. To this end, we present DGD, a unified 3D representation for both the appearance and semantics of a dynamic 3D scene, building upon the recently proposed dynamic 3D Gaussians representation. Our representation is optimized over time with both color and semantic information. Key to our method is the joint optimization of the appearance and semantic attributes, which jointly affect the geometric properties of the scene. We evaluate our approach in its ability to enable dense semantic 3D object tracking and demonstrate high-quality results that are fast to render, for a diverse set of scenes.

我们处理的任务是基于单个单目视频学习动态三维语义辐射场。我们学习到的语义辐射场能够捕捉每个点的语义以及动态三维场景的颜色和几何特性,从而生成新视角及其对应的语义。这使得可以使用简单直观的界面(如用户点击或文本提示)对各种三维语义实体进行分割和跟踪。为此,我们提出了DGD,这是一种用于动态三维场景外观和语义的统一三维表示,基于最近提出的动态三维高斯表示构建。我们的表示随时间进行优化,结合了颜色和语义信息。我们方法的关键是外观和语义属性的联合优化,这共同影响场景的几何特性。我们在密集语义三维对象跟踪能力方面对我们的方法进行了评估,并展示了在各种场景中快速渲染的高质量结果。