Lune

CVPR2022Top-tier venue

GeoNeRF: Generalizing NeRF with Geometry Priors

Mohammad Mahdi Johari, Yann Lepoittevin, François Fleuret

Abstract

We present GeoNeRF, a generalizable photorealistic novel view synthesis method based on neural radiance fields. Our approach consists of two main stages: a ge-ometry reasoner and a renderer. To render a novel view, the geometry reasoner first constructs cascaded cost volumes for each nearby source view. Then, using a Transformer- based attention mechanism and the cascaded cost volumes, the renderer infers geometry and appearance, and ren-ders detailed images via classical volume rendering techniques. This architecture, in particular, allows sophis-ticated occlusion reasoning, gathering information from consistent source views. Moreover, our method can eas-ily be fine-tuned on a single scene, and renders com-petitive results with per-scene optimized neural rendering methods with a fraction of computational cost. Ex-periments show that GeoNeRF outperforms state-of-the- art generalizable neural rendering models on various syn-thetic and real datasets. Lastly, with a slight modification to the geometry reasoner, we also propose an alter-native model that adapts to RGBD images. This model di-rectly exploits the depth information often available thanks to depth sensors. The implementation code is available at https://www.idiap.ch/paper/geonerf.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

90 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization cites NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
  • ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views cites One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization
  • MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction Model cites One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers90

  1. One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape OptimizationMinghua Liu, Chao Xu, Haian Jin, Linghao Chen et al.NeurIPS 2023 · 755 citations
  2. ScanNet++: A High-Fidelity Dataset of 3D Indoor ScenesChandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela DaiICCV 2023 · 659 citations
  3. MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse ViewsYuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang et al.NeurIPS 2024 · 126 citations
  4. DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian ConsensusYu Chen, Gim Hee LeeNeurIPS 2024 · 99 citations
  5. MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction ModelMinghua Liu, Chong Zeng, Xinyue Wei, Ruoxi Shi et al.NeurIPS 2024 · 73 citations

Builds on26

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman et al.ICCV 2021 · 2,700 citations
  3. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view ReconstructionPeng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt et al.NeurIPS 2021 · 2,500 citations
  4. Nerfies: Deformable Neural Radiance FieldsKeunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz et al.ICCV 2021 · 1,442 citations
  5. Volume Rendering of Neural Implicit SurfacesLior Yariv, Jiatao Gu, Yoni Kasten, Yaron LipmanNeurIPS 2021 · 1,421 citations

Related papers

  1. Is Attention All That NeRF Needs?Mukund Varma T., Peihao Wang, Xuxi Chen, Tianlong Chen et al.ICLR 2023 · 6 citations
  2. GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene UnderstandingZi-Ting Chou, Sheng-Yu Huang, I-Jieh Liu, Yu-Chiang Frank WangCVPR 2024
  3. GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View SynthesisYou Wang, Li Fang, Hao Zhu, Fei Hu et al.CVPR 2025
  4. MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View StereoAnpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang et al.ICCV 2021 · 1,024 citations
  5. GM-NeRF: Learning Generalizable Model-Based Neural Radiance Fields from Multi-View ImagesJianchuan Chen, Wentao Yi, Liqian Ma, Xu Jia et al.CVPR 2023
Dusk over the sea between two cliffs drawn in fine vertical lines