HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios
Xiaochao Pan, Jiawei Yao, Hongrui Kou, Tong Wu, Canran Xiao
Abstract
In the realm of autonomous driving, achieving precise 3D reconstruction of the driving environment is critical for ensuring safety and effective navigation. Neural Radiance Fields (NeRF) have shown promise in creating highly detailed and accurate models of complex environments. However, the application of NeRF in autonomous driving scenarios encounters several challenges, primarily due to the sparsity of viewpoints inherent in camera trajectories and the constraints on data collection in unbounded outdoor scenes, which typically occur along predetermined paths. This limitation not only reduces the available scene information but also poses significant challenges for NeRF training, as the sparse and path-distributed observational data leads to under-representation of the scene's geometry. In this paper, we introduce HarmonicNeRF, a novel approach for outdoor self-supervised monocular scene reconstruction. HarmonicNeRF capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. This is achieved through the application of spherical harmonics to generate novel radiance values, taking into careful consideration the color observations from the limited available real-world views. Additionally, our method incorporates proxy geometry to effectively manage occlusion, generating radiance pseudo-labels that circumvent the limitations of traditional image-warping techniques, which often fail in sparse data conditions typical of autonomous driving environments. Extensive experiments conducted on the KITTI, Argoverse, and NuScenes datasets demonstrate our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods. Project page: https://github.com/Jiawei-Yao0812/HarmonicNeRF
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Dehallu3D: Hallucination-Mitigated 3D Generation from a Single Image via Cyclic View Consistency Refinement cites Instant neural graphics primitives with a multiresolution hash encoding
- Path Matters: Unveiling Geometric Implicit Bias via Curvature-Aware Sparse View Optimization cites NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
- Path Matters: Unveiling Geometric Implicit Bias via Curvature-Aware Sparse View Optimization cites MonoFusion: Sparse-View 4D Reconstruction via Monocular Fusion
- Path Matters: Unveiling Geometric Implicit Bias via Curvature-Aware Sparse View Optimization cites Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
- Instant neural graphics primitives with a multiresolution hash encoding cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Cited by top-tier papers4
- Confusion-Resistant Federated Learning via Diffusion-Based Data Harmonization on Non-IID DataXiaohong Chen, Canran Xiao, Yongmei LiuNeurIPS 2024 · 41 citations
- MonoFusion: Sparse-View 4D Reconstruction via Monocular FusionZihan Wang, Jeff Tan, Tarasha Khurana, Neehar Peri et al.ICCV 2025 · 5 citations
- Dehallu3D: Hallucination-Mitigated 3D Generation from a Single Image via Cyclic View Consistency RefinementXiwen Wang, Shichao Zhang, Ruowei Wang, Mao Li et al.CVPR 2026
- Path Matters: Unveiling Geometric Implicit Bias via Curvature-Aware Sparse View OptimizationCanran Xiao, Liaoyuan Fan, Yanbin Li, Jing Tang et al.ICLR 2026
Builds on41
- Instant neural graphics primitives with a multiresolution hash encodingThomas Müller, Alex Evans, Christoph Schied, Alexander KellerSIGGRAPH 2022 · 4,089 citations
- Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman et al.ICCV 2021 · 2,700 citations
- NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view ReconstructionPeng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt et al.NeurIPS 2021 · 2,500 citations
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan et al.CVPR 2022 · 1,603 citations
- Volume Rendering of Neural Implicit SurfacesLior Yariv, Jiatao Gu, Yoni Kasten, Yaron LipmanNeurIPS 2021 · 1,421 citations
Related papers
- AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose OptimizationKun Wang, Zhiqiang Yan, Huang Tian, Zhenyu Zhang et al.AAAI 2024 · 6 citations
- SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance FieldsAnh-Quan Cao, Raoul de CharetteICCV 2023 · 73 citations
- S-NeRF: Neural Radiance Fields for Street ViewsZiyang Xie, Junge Zhang, Wenye Li, Feihu Zhang et al.ICLR 2023 · 13 citations
- LidaRF: Delving into Lidar for Neural Radiance Field on Street ScenesShanlin Sun, Bingbing Zhuang, Ziyu Jiang, Buyu Liu et al.CVPR 2024 · 11 citations
- MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera PosesYang Fu, Ishan Misra, Xiaolong WangICML 2023 · 13 citations
