Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

Volume rendering by rasterizing sparse voxels. (b) Novel-view rendering on Mip-NeRF360 scenes. (c) 2D-to-3D made easy. 3DGS variants NeRF variants Neural-free voxel grids * The actual voxel sizes are much smaller. Ours 2D VFM feature 3D VFM feature 2D semantic 3D semantic TSDF Fusion Marching Cubes *Our FPS comparison to 3DGS is highly scene dependent. Figure 1. We propose SVRaster, a novel framework for multi-view reconstruction and novel view synthesis. (a) Sparse voxel representation effectively captures the volume density and radiance field of the scene, without the need for neural networks, 3D Gaussians, and sparse-points prior. (b) Using our customized sparse voxel rasterizer, we can learn the underlying 3D scene efficiently and achieve state-ofthe-art performance in both rendering quality and speed. (c) Notably, lifting 2D modal to the trained sparse voxels is simple and efficient by integrating the classic Volume Fusion [7, 8, 34]. We show examples of vision foundation model feature field from RADIO [38], semantic field from Segformer [52], and signed distance field from rendered depth, making it flexible and suitable for a wide range of applications.