Video-Guided Foley Sound Generation with Multimodal Controls
Ziyang Chen, Prem Seetharaman, Bryan C. Russell, Oriol Nieto, David Bourgin, Andrew Owens, Justin Salamon
2025Year
25Top-tier citations
Abstract
2 Adobe Research https://ificl.github.io/MultiFoley/ (a) Foley with text control (b) Foley with audio control (c) Foley audio extension + "lion roaring" -"cat meowing"
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 239d4452-d2c9-44bf-9d96-5a12d05b0dfbCited by top-tier papers25
- AudioX: A Unified Framework for Anything-to-Audio GenerationZeyue Tian, Zhaoyang Liu, Yizhu Jin, Ruibin Yuan et al.ICLR 2026 · 38 citations
- Omni2Sound: Towards Unified Video-Text-to-Audio Generationyusheng dai, Zehua Chen, Yuxuan Jiang, Qiuhong Ke et al.CVPR 2026 · 12 citations
- SmartDJ: Declarative Audio Editing with Audio Language ModelZitong Lan, Yiduo Hao, Mingmin ZhaoICLR 2026 · 11 citations
- VAFlow: Video-to-Audio Generation with Cross-Modality Flow MatchingXihua Wang, Xin Cheng, Yuyue Wang, Ruihua Song et al.ICCV 2025 · 6 citations
- ThinkSound: Chain-of-Thought Reasoning in Multimodal LLMs for Audio Generation and EditingHuadai Liu, Kaicheng Luo, Jialei Wang, Wen Wang et al.NeurIPS 2025 · 5 citations
Builds on44
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 5,568 citations
Related papers
- CAFA: A Controllable Automatic Foley ArtistRoi Benita, Michael Finkelson, Tavi Halperin, Gleb Sterkin et al.ICCV 2025 · 1 citation
- AC-Foley: Reference-Audio-Guided Video-to-Audio Synthesis with Acoustic TransferPengjun Fang, Yingqing He, Yazhou Xing, Qifeng Chen et al.ICLR 2026 · 3 citations
- Tri-Ergon: Fine-Grained Video-to-Audio Generation with Multi-Modal Conditions and LUFS ControlBingliang Li, Fengyu Yang, Yuxin Mao, Qingwen Ye et al.AAAI 2025 · 3 citations
- EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound GenerationBingxuan Li, Yiming Cui, Yicheng He, Yiwei Wang et al.CVPR 2026 · 5 citations
- FoleyDirector: Fine-Grained Temporal Steering for Video-to-Audio Generation via Structured ScriptsYou Li, Dewei Zhou, Fan Ma, Fu Li et al.CVPR 2026 · 2 citations
