FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation
Chenhan Jiang, Yu Chen, Qingwen Zhang, Jifei Song, Songcen Xu, Dit-Yan Yeung, Jiankang Deng
摘要
The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data with diverse and accurate camera trajectories. While real-world captures are photorealistic, they are typically sparse and discrete. Conversely, synthetic data scales but suffers from a domain gap and often lacks realistic semantics. We introduce FVGen, a novel framework that leverages the power of scene reconstruction to transform limited real-world image sequences into a scalable source of high-quality training data. Our key insight is that an imperfect reconstructed scene serves as a rich geometric proxy, but naively sampling from it amplifies artifacts. To this end, we propose a certainty-aware free-view sampling strategy that identifies novel viewpoints which are both semantically meaningful and minimally affected by reconstruction errors. We demonstrate FVGen's effectiveness by scaling up the training of feedforward NVS models, achieving a significant improvement of 2.6 dB on challenging out-of-distribution benchmarks. Furthermore, we show that the generated data can actively enhance per-scene 3D Gaussian Splatting optimization, leading to consistent improvements across multiple datasets. Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper31
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 被引用 5,687 次
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan 等CVPR 2022 · 被引用 1,603 次
- Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene UnderstandingMike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar 等ICCV 2021 · 被引用 633 次
- Dense Depth Priors for Neural Radiance Fields from Sparse Input ViewsBarbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan 等CVPR 2022 · 被引用 319 次
- DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii 等CVPR 2024 · 被引用 302 次
相关 Paper
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor ScenesYunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee LeeNeurIPS 2024 · 被引用 112 次
- FlowR: Flowing from Sparse to Dense 3D ReconstructionsTobias Fischer, Samuel Rota Bulò, Yung-Hsu Yang, Nikhil Varma Keetha 等ICCV 2025 · 被引用 6 次
- ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion PriorsMinsu Kim, Subin Jeon, In Cho, Mijin Yoo 等ICCV 2025 · 被引用 1 次
- VGNC: Reducing the Overfitting of Sparse-view 3DGS via Validation-guided Gaussian Number ControlLifeng Lin, Rongfeng Lu, Quan Chen, Haofan Ren 等ACM MM 2025 · 被引用 2 次
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse ViewsYuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang 等NeurIPS 2024 · 被引用 126 次
