Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization
Jiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang, Jun Xue
摘要
Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under single-sided or asynchronous forgeries propagates cross-modal noise, degrading high-precision localization. We present IaMSB, an inconsistency-aware multimodal Schrödinger Bridge (SB) that jointly estimates cross-modal consistency and performs interval-level localization. Unlike diffusion models, SB minimizes path-distribution discrepancy and yields consistency scores without explicit noise injection or denoising. With the Schrödinger Bridge (SB), IaMSB unifies consistency estimation, cross-modal information selection, and bridge-step scheduling in one framework. Specifically, a lightweight coarse bridge first proposes candidate intervals and estimates cross-modal consistency; these statistics select cross-modal witness signals and allocate bridge steps asymmetrically across modalities. A refinement bridge then performs step-tuned fusion and outputs refined, time-aligned intervals. IaMSB anticipates single-sided and asynchronous forgeries and, using bottlenecked cross-modal interaction with step allocation, suppresses noise transfer, avoids unnecessary iterations. Across benchmarks, IaMSB stabilizes strict-IoU boundary precision, raising AP@0.95 by , and yields improved high-precision localization, particularly for single-sided forgeries.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper20
- VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-TrainingZhan Tong, Yibing Song, Jue Wang, Limin WangNeurIPS 2022 · 被引用 2,336 次
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative ModelingValentin De Bortoli, James Thornton, Jeremy Heng, Arnaud DoucetNeurIPS 2021 · 被引用 811 次
- Learning Audio-Visual Speech Representation by Masked Multimodal Cluster PredictionBowen Shi, Wei-Ning Hsu, Kushal Lakhotia, Abdelrahman MohamedICLR 2022 · 被引用 460 次
- Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and LocalizationKomal Chugh, Parul Gupta, Abhinav Dhall, Ramanathan SubramanianACM MM 2020 · 被引用 217 次
- MAViL: Masked Audio-Video LearnersPo-Yao Huang, Vasu Sharma, Hu Xu, Chaitanya Ryali 等NeurIPS 2023 · 被引用 95 次
相关 Paper
- Structural–Semantic Perception for Diffusion-Guided Temporal Forgery LocalizationLigong Cao, Yeting Guo, Haoang ChiCVPR 2026
- Intra-Modal and Cross-Modal Synchronization for Audio-Visual Deepfake Detection and Temporal LocalizationAshutosh Anshul, Shreyas Gopal, Deepu Rajan, Eng Siong ChngICCV 2025 · 被引用 10 次
- A Multimodal Deviation Perceiving Framework for Weakly-Supervised Temporal Forgery LocalizationWenbo Xu, Junyan Wu, Wei Lu, Xiangyang Luo 等ACM MM 2025 · 被引用 2 次
- Bridging Pixels and Words: Mask-Aware Local Semantic Fusion for Multimodal Media VerificationZizhao Chen, Ping Wei, Ziyang Ren, Huan Li 等CVPR 2026
- Frequency-Dependent Scheduled Schrödinger Bridge for Underwater Acoustic Signal DenoisingPengsen Zhu, Lina Gao, Yulong Huang, Lifeng Liu 等AAAI 2026
