UE4-NeRF: Neural Radiance Field for Real-Time Rendering of Large-Scale Scene

Neural Radiance Field (NeRF) is an implicit 3D reconstruction method that has shown immense potential and has gained significant attention for its ability to reconstruct 3D scenes solely from a set of photographs. However, its real-time rendering capability, especially for interactive real-time rendering of large-scale scenes, has significant limitations. To address this challenge, we propose a novel neural rendering system called UE4-NeRF that is designed for real-time rendering of large-scale scenes. Our proposed approach partitions large scenes into sub-NeRFs, and uses polygonal meshes to represent them. In order to represent the partitioned independent scene, we initialize polygonal meshes by constructing multiple regular octahedra within the scene and the vertices of the polygonal faces are continuously optimized during the training process. Drawing inspiration from the Level of Detail (LOD) techniques, we train meshes with varying levels of detail for different observation levels. Our approach combines with the rasterization pipeline in Unreal Engine 4 (UE4), achieving real-time rendering of large-scale scenes at 4K resolution with a frame rate of up to 43 FPS. Our experimental results demonstrate that our method attains rendering quality on par with state-of-the-art approaches, while additionally offering the advantage of real-time performance. Furthermore, rendering within UE4 facilitates scene editing in subsequent stages. Project page: https://jamchaos.github.io/UE4-NeRF/ .