Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, Pratul P. Srinivasan
Abstract
The rendering procedure used by neural radiance fields (NeRF) samples a scene with a single ray per pixel and may therefore produce renderings that are excessively blurred or aliased when training or testing images observe scene content at different resolutions. The straightforward solution of supersampling by rendering with multiple rays per pixel is impractical for NeRF, because rendering each ray requires querying a multilayer perceptron hundreds of times. Our solution, which we call "mip-NeRF" (à la "mipmap"), extends NeRF to represent the scene at a continuously-valued scale. By efficiently rendering anti-aliased conical frustums instead of rays, mip-NeRF reduces objectionable aliasing artifacts and significantly improves NeRF’s ability to represent fine details, while also being 7% faster than NeRF and half the size. Compared to NeRF, mip-NeRF reduces average error rates by 17% on the dataset presented with NeRF and by 60% on a challenging multiscale variant of that dataset that we present. Mip-NeRF is also able to match the accuracy of a brute-force supersampled NeRF on our multiscale dataset while being 22× faster.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
815 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction cites GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction cites Neural Sparse Voxel Fields
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction cites Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Instant neural graphics primitives with a multiresolution hash encoding cites Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
- Instant neural graphics primitives with a multiresolution hash encoding cites Efficient Geometry-aware 3D Generative Adversarial Networks
- Instant neural graphics primitives with a multiresolution hash encoding cites Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Instant neural graphics primitives with a multiresolution hash encoding
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Plenoxels: Radiance Fields without Neural Networks
- Efficient Geometry-aware 3D Generative Adversarial Networks cites GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
- Efficient Geometry-aware 3D Generative Adversarial Networks cites Neural Sparse Voxel Fields
- Efficient Geometry-aware 3D Generative Adversarial Networks cites Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Efficient Geometry-aware 3D Generative Adversarial Networks cites Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
- Plenoxels: Radiance Fields without Neural Networks cites Neural Sparse Voxel Fields
Cited by top-tier papers815
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- Instant neural graphics primitives with a multiresolution hash encodingThomas Müller, Alex Evans, Christoph Schied, Alexander KellerSIGGRAPH 2022 · 4,089 citations
- Plenoxels: Radiance Fields without Neural NetworksSara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen et al.CVPR 2022 · 1,237 citations
- Efficient Geometry-aware 3D Generative Adversarial NetworksEric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano et al.CVPR 2022 · 984 citations
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields ReconstructionCheng Sun, Min Sun, Hwann-Tzong ChenCVPR 2022 · 859 citations
Builds on13
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional DomainsMatthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil et al.NeurIPS 2020 · 4,036 citations
- Neural Sparse Voxel FieldsLingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua et al.NeurIPS 2020 · 1,535 citations
- Multiview Neural Surface Reconstruction by Disentangling Geometry and AppearanceLior Yariv, Yoni Kasten, Dror Moran, Meirav Galun et al.NeurIPS 2020 · 1,010 citations
- GRAF: Generative Radiance Fields for 3D-Aware Image SynthesisKatja Schwarz, Yiyi Liao, Michael Niemeyer, Andreas GeigerNeurIPS 2020 · 1,001 citations
- NeuMIP: multi-resolution neural materialsAlexandr Kuznetsov, Krishna Mullia, Zexiang Xu, Milos Hasan et al.SIGGRAPH 2021 · 68 citations
Related papers
- Mip-Grid: Anti-aliased Grid Representations for Neural Radiance FieldsSeungtae Nam, Daniel Rho, Jong Hwan Ko, Eunbyung ParkNeurIPS 2023 · 22 citations
- PyNeRF: Pyramidal Neural Radiance FieldsHaithem Turki, Michael Zollhöfer, Christian Richardt, Deva RamananNeurIPS 2023 · 31 citations
- Multiscale Representation for Real-Time Anti-Aliasing Neural RenderingDongting Hu, Zhenkai Zhang, Tingbo Hou, Tongliang Liu et al.ICCV 2023 · 16 citations
- Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance FieldsWenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang et al.ICCV 2023 · 182 citations
- Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan et al.ICCV 2023 · 799 citations
