Recently, Gaussian splatting has emerged as a robust technique for representing 3D scenes, enabling real-time rasterization and high-fidelity rendering. However, Gaussians' inherent radial symmetry and smoothness constraints limit their ability to represent complex shapes, often requiring thousands of primitives to approximate detailed geometry. We introduce Deformable Radial Kernel (DRK), which extends Gaussian splatting into a more general and flexible framework. Through learnable radial bases with adjustable angles and scales, DRK efficiently models diverse shape primitives while enabling precise control over edge sharpness and boundary curvature. iven DRK's planar nature, we further develop accurate ray-primitive intersection computation for depth sorting and introduce efficient kernel culling strategies for improved rasterization efficiency. Extensive experiments demonstrate that DRK outperforms existing methods in both representation efficiency and rendering quality, achieving state-of-the-art performance while dramatically reducing primitive count.
Our DRK representation extends Gaussian kernels with learnable radial bases, enabling the modeling of diverse shape primitives. Additionally, parameters are introduced to control the sharpness and boundary curvature of these primitives.
The rasterization process of DRK begins by (1) computing the intersection of rays with the DRK tangent planes. Next, (2) the scaling factor is deformed using the rotational radial bases. (3) The L1 and L2 norms of the intersection point relative to the kernel center are then calculated and interpolated to match the target curvature. Finally, (4) the boundary of the DRK is sharpened using a learnable sharpness parameter to obtain the final alpha value for rendering.
The following video showcase the effectiveness of each parameter.
3D-GS employs EWA projection, which approximates Gaussian depth using the center depth. However, this can introduce artifacts when the depth ordering of Gaussian centers does not match the true depth ordering of the Gaussians themselves. To overcome this limitation, we propose using a sorted cache array to store DRK kernels, allowing the nearest kernel to be efficiently selected and rasterized. This method significantly improves depth sorting accuracy and eliminates artifacts.
Leveraging the explicit formulation of the DRK function, we can directly compute kernel endpoints and estimate a bounding polygon for each kernel. This enables efficient culling of DRK kernels on non-overlapping tiles, significantly enhancing rasterization performance.
We demonstrate the flexibility of DRK in fitting complex shapes using diverse primitives. The following video highlights DRK's fitting results on various shapes.
Thanks to the flexibility of DRK, triangle/polygon meshes can be converted into DRK representations without requiring training views, rendering, or model optimization. This enables the seamless conversion of millions of 3D assets into DRK representations for real-time rendering. The following video showcases the conversion results.
Train-free conversion from meshes to DRK enables millions of 3D assets to seamlessly integrate into high-fidelity reconstructed real scenes. Both types of assets are rendered together within a unified rendering pipeline. The following video showcases the resulting composed renderings.
Deformable Radial Kernel Splatting
@article{huang2024deformable,
title={Deformable Radial Kernel Splatting},
author={Huang, Yi-Hua and Lin, Ming-Xian and Sun, Yang-Tian and Yang, Ziyi and Lyu, Xiaoyang and Cao, Yan-Pei and Qi, Xiaojuan},
journal={arXiv preprint arXiv:2412.11752},
year={2024}
}There are some excellent works that highly related to our work. We list some of them here.
We sincerely thank BinBin Huang for his valuable suggestions and feedbacks. He is an excellent researcher in the fields of 3D representation and reconstruction.
The website template is borrowed from Nerfies.