FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation
Chenhan Jiang, Yu Chen, Qingwen Zhang, Jifei Song, Songcen Xu, Dit-Yan Yeung, Jiankang Deng
Abstract
The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data with diverse and accurate camera trajectories. While real-world captures are photorealistic, they are typically sparse and discrete. Conversely, synthetic data scales but suffers from a domain gap and often lacks realistic semantics. We introduce FVGen, a novel framework that leverages the power of scene reconstruction to transform limited real-world image sequences into a scalable source of high-quality training data. Our key insight is that an imperfect reconstructed scene serves as a rich geometric proxy, but naively sampling from it amplifies artifacts. To this end, we propose a certainty-aware free-view sampling strategy that identifies novel viewpoints which are both semantically meaningful and minimally affected by reconstruction errors. We demonstrate FVGen's effectiveness by scaling up the training of feedforward NVS models, achieving a significant improvement of 2.6 dB on challenging out-of-distribution benchmarks. Furthermore, we show that the generated data can actively enhance per-scene 3D Gaussian Splatting optimization, leading to consistent improvements across multiple datasets. Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on31
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan et al.CVPR 2022 · 1,603 citations
- Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene UnderstandingMike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar et al.ICCV 2021 · 633 citations
- Dense Depth Priors for Neural Radiance Fields from Sparse Input ViewsBarbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan et al.CVPR 2022 · 319 citations
- DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii et al.CVPR 2024 · 302 citations
Related papers
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor ScenesYunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee LeeNeurIPS 2024 · 112 citations
- FlowR: Flowing from Sparse to Dense 3D ReconstructionsTobias Fischer, Samuel Rota Bulò, Yung-Hsu Yang, Nikhil Varma Keetha et al.ICCV 2025 · 6 citations
- ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion PriorsMinsu Kim, Subin Jeon, In Cho, Mijin Yoo et al.ICCV 2025 · 1 citation
- VGNC: Reducing the Overfitting of Sparse-view 3DGS via Validation-guided Gaussian Number ControlLifeng Lin, Rongfeng Lu, Quan Chen, Haofan Ren et al.ACM MM 2025 · 2 citations
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse ViewsYuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang et al.NeurIPS 2024 · 126 citations
