Geometry-based Schrödinger Bridges for Trustworthy Multimodal Fusion
Jiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang, Jun Xue
Abstract
Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a model is confident but wrong (overconfident), these methods fail to detect the error. To break this loop, we propose Geometry-based Multimodal Fusion (GMF). Instead of relying on predictions, we evaluate reliability by measuring the physical effort required to map input data back to the valid data manifold. We implement this using Diffusion Schrödinger Bridges with Rectified Flow, which allows us to calculate Transport Energy as a direct metric for quality. The logic is simple: valid data sits on the manifold (low energy), while noisy, incomplete data or conflicting data requires high energy to be restored. This geometric metric acts as an independent judge, effectively flagging unreliable inputs even when the classifier is fooled. Extensive experiments demonstrate that GMF significantly improves robustness against severe sensor noise and semantic conflicts compared to confidence-based baselines.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 0f0aa837-1f1b-4c85-aa93-7d00d719a131Builds on10
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative ModelingValentin De Bortoli, James Thornton, Jeremy Heng, Arnaud DoucetNeurIPS 2021 · 811 citations
- Diffusion Schrödinger Bridge MatchingYuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud DoucetNeurIPS 2023 · 178 citations
- Are Multimodal Transformers Robust to Missing Modality?Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine et al.CVPR 2022 · 153 citations
Related papers
- Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake LocalizationJiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang et al.CVPR 2026
- Structure-Aware Fusion with Progressive Injection for Multimodal Molecular Representation LearningZihao Jing, Yan Sun, Yan Yi Li, Sugitha Janarthanan et al.NeurIPS 2025 · 2 citations
- Bridging the Modality Reliability Gap in Drug-Target Interaction Prediction via a Confidence-aware Multimodal Fusion FrameworkJie Yang, Junxiong Zhang, Kun Qian, Qingyu Yang et al.AAAI 2026
- Gaussian Fusion: Accurate 3D Reconstruction via Geometry-Guided Displacement InterpolationDuo Chen, Zixin Tang, Zhenyu Xu, Yunan Zheng et al.ICCV 2021 · 4 citations
- Provable Dynamic Fusion for Low-Quality Multimodal DataQingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu et al.ICML 2023 · 143 citations
