Geometry-based Schrödinger Bridges for Trustworthy Multimodal Fusion
Jiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang, Jun Xue
摘要
Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a model is confident but wrong (overconfident), these methods fail to detect the error. To break this loop, we propose Geometry-based Multimodal Fusion (GMF). Instead of relying on predictions, we evaluate reliability by measuring the physical effort required to map input data back to the valid data manifold. We implement this using Diffusion Schrödinger Bridges with Rectified Flow, which allows us to calculate Transport Energy as a direct metric for quality. The logic is simple: valid data sits on the manifold (low energy), while noisy, incomplete data or conflicting data requires high energy to be restored. This geometric metric acts as an independent judge, effectively flagging unreliable inputs even when the classifier is fooled. Extensive experiments demonstrate that GMF significantly improves robustness against severe sensor noise and semantic conflicts compared to confidence-based baselines.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 被引用 2,213 次
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative ModelingValentin De Bortoli, James Thornton, Jeremy Heng, Arnaud DoucetNeurIPS 2021 · 被引用 811 次
- Diffusion Schrödinger Bridge MatchingYuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud DoucetNeurIPS 2023 · 被引用 178 次
- Are Multimodal Transformers Robust to Missing Modality?Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine 等CVPR 2022 · 被引用 153 次
相关 Paper
- Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake LocalizationJiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang 等CVPR 2026
- Structure-Aware Fusion with Progressive Injection for Multimodal Molecular Representation LearningZihao Jing, Yan Sun, Yan Yi Li, Sugitha Janarthanan 等NeurIPS 2025 · 被引用 2 次
- Bridging the Modality Reliability Gap in Drug-Target Interaction Prediction via a Confidence-aware Multimodal Fusion FrameworkJie Yang, Junxiong Zhang, Kun Qian, Qingyu Yang 等AAAI 2026
- Gaussian Fusion: Accurate 3D Reconstruction via Geometry-Guided Displacement InterpolationDuo Chen, Zixin Tang, Zhenyu Xu, Yunan Zheng 等ICCV 2021 · 被引用 4 次
- Provable Dynamic Fusion for Low-Quality Multimodal DataQingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu 等ICML 2023 · 被引用 143 次
