SonoWorld: From One Image to a 3D Audio-Visual Scene
Derong Jin, Xiyi Chen, Ming C. Lin, Ruohan Gao
Abstract
Tremendous progress in visual scene generation now turns a single image into an explorable 3D world, yet immersion remains incomplete without sound. We introduce Image2AVScene, the task of generating a 3D audio-visual scene from a single image, and present SonoWorld, the first framework to tackle this challenge. From one image, our pipeline outpaints a 360 panorama, lifts it into a navigable 3D scene, places language-guided sound anchors, and renders ambisonics for point, areal, and ambient sources, yielding spatial audio aligned with scene geometry and semantics. Quantitative evaluations on a newly curated real-world dataset and a controlled user study confirm the effectiveness of our approach. Beyond free-viewpoint audio-visual rendering, we also demonstrate applications to one-shot acoustic learning and audio-visual spatial source separation. Project website: https://humathe.github.io/sonoworld/
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on37
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- SAM 3: Segment Anything with ConceptsNicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath et al.ICLR 2026 · 1,103 citations
- AudioLDM: Text-to-Audio Generation with Latent Diffusion ModelsHaohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei et al.ICML 2023 · 773 citations
- Depth Anything 3: Recovering the Visual Space from Any ViewsHaotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen et al.ICLR 2026 · 720 citations
- CAT3D: Create Anything in 3D with Multi-View Diffusion ModelsRuiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee et al.NeurIPS 2024 · 490 citations
Related papers
- Image as a World: Generating Interactive World from Single Image via Panoramic Video GenerationDongnan Gui, Xun Guo, Wengang Zhou, Yan LuNeurIPS 2025 · 5 citations
- Sonic4D: Spatial Audio Generation for Immersive 4D Scene ExplorationSiyi Xie, Hanxin Zhu, Xinyi Chen, Tianyu He et al.AAAI 2026 · 4 citations
- One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single ImagePengfei Wang, Liyi Chen, Zhiyuan Ma, Yanjun Guo et al.ICLR 2026 · 11 citations
- Sound to Visual Scene Generation by Audio-to-Visual Latent AlignmentSung-Bin Kim, Arda Senocak, Hyunwoo Ha, Andrew Owens et al.CVPR 2023
- A Recipe for Generating 3D Worlds from a Single ImageKatja Schwarz, Denis Rozumny, Samuel Rota Bulò, Lorenzo Porzi et al.ICCV 2025 · 5 citations
