We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex appearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.
我们提出了一种基于空间和角度高斯表示的三重散射过程，用于从多视角点光源输入图像中进行实时、高质量的新光照和视角合成。为了描述复杂的外观，我们为每个空间高斯采用了朗伯反射与混合角度高斯的有效反射函数。为了生成自阴影，我们将所有空间高斯投影到光源方向以获取阴影值，这些值随后通过一个小型多层感知机（MLP）进行进一步优化。为了补偿全局光照等其他效果，我们训练了另一个网络来计算并为每个空间高斯添加RGB三元组。我们的表示在几何形状（从实心到松软）和外观（从半透明到各向异性）变化广泛的30个样本上得到了验证，并使用了不同形式的输入数据，包括合成/重建对象的渲染图像、手持相机加闪光灯拍摄的照片，以及专业光照台的图像。我们在一块普通GPU上实现了40到70分钟的训练时间和每秒90帧的渲染速度。我们的结果在质量和性能方面与最先进的技术相比表现优越。