3D Gaussian Splatting (3D-GS) technique couples 3D Gaussian primitives with differentiable rasterization to achieve high-quality novel view synthesis results while providing advanced real-time rendering performance. However, due to the flaw of its adaptive density control strategy in 3D-GS, it frequently suffers from over-reconstruction issue in intricate scenes containing high-frequency details, leading to blurry rendered images. The underlying reason for the flaw has still been under-explored. In this work, we present a comprehensive analysis of the cause of aforementioned artifacts, namely gradient collision, which prevents large Gaussians in over-reconstructed regions from splitting. To address this issue, we propose the novel homodirectional view-space positional gradient as the criterion for densification. Our strategy efficiently identifies large Gaussians in over-reconstructed regions, and recovers fine details by splitting. We evaluate our proposed method on various challenging datasets. The experimental results indicate that our approach achieves the best rendering quality with reduced or similar memory consumption. Our method is easy to implement and can be incorporated into a wide variety of most recent Gaussian Splatting-based methods.
Pipeline of AbsGS. (a) The splat-based rendering technique projects Gaussian \( G_i \) with mean position \( \mu_{i}^{3d} \) to 2D coordinate \( \mu_{i} \) in pixel-space. The number of covered pixels by Gaussian \( G_i \) is \( m \). (b) By backpropagating, the view-space gradient \( g_i \) of Gaussian \( G_i \) under viewpoint \( k \) is calculated as the sum of all view-space gradients of pixels that are covered by \( G_i \). Since the gradients \( \frac{\partial L_j}{\partial \mu_{i}} \) have different directions, the overall sum \( g_i \) will have a small scale, which does not satisfy the gradient threshold for densification. (c) Motivated by the above analysis, we redesign the densification scheme by taking the absolute value of each component \( \lvert \frac{\partial L_j}{\partial \mu_{i,x}} \rvert \) and \( \lvert \frac{\partial L_j}{\partial \mu_{i,y}} \rvert \) before summing. This enables us to identify large Gaussians in over-reconstructed regions for split.
We present the NVS results of 3D-GS, AbsGS (Ours), and Ground Truth.
We present the point clouds by extracting mean positions of Gaussian primitives.
We train the treehill scene with 3D-GS using default parameters for 7000 steps; then we present selected Gaussian ellipsoids for densifcation by 3D-GS and ours, with green representing splitting and blue representing cloning. Remarkably, the number of selected Gaussians between the two methods remained comparable, 69,787 and 52,517 Gaussians for 3D-GS and our method respectively.
@misc{ye2024absgs,
title={AbsGS: Recovering Fine Details for 3D Gaussian Splatting},
author={Zongxin Ye and Wenyu Li and Sidun Liu and Peng Qiao and Yong Dou},
year={2024},
eprint={2404.10484},
archivePrefix={arXiv},
primaryClass={cs.CV}
}