Lune

CVPR Findings2026

Unbiased Dynamic Multimodal Fusion

Shicai Wei, Kaijie Zhang, Luyi Chen, Tao He, Guiduo Duan

Abstract

Traditional multimodal methods often assume static modality quality, which limits their adaptability in dynamic realworld scenarios. Thus, dynamical multimodal methods are proposed to assess modality quality and adjust their contribution accordingly. However, they typically rely on empirical metrics, failing to measure the modality quality when noise levels are extremely low or high. Moreover, existing methods usually assume that the initial contribution of each modality is the same, neglecting the intrinsic modality dependency bias. As a result, the modality hard to learn would be doubly penalized, and the performance of dynamical fusion could be inferior to that of static fusion. To address these challenges, we propose the Unbiased Dynamic Multimodal Learning (UDML) framework. Specifically, we introduce a noise-aware uncertainty estimator that adds controlled noise to the modality data and predicts its intensity from the modality feature. This forces the model to learn a clear correspondence between feature corruption and noise level, allowing accurate uncertainty measure across both low-and high-noise conditions. Furthermore, we quantify the inherent modality reliance bias within multimodal networks via modality dropout and incorporate it into the weighting mechanism. This eliminates the dual suppression effect on the hard-to-learn modality. Extensive experiments across diverse multimodal benchmark tasks validate the effectiveness, versatility, and generalizability of the proposed UDML. The code is available at https://github.com/shicaiwei123/UDML.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Provable Dynamic Fusion for Low-Quality Multimodal Data cites Balanced Multimodal Learning via On-the-fly Gradient Modulation
  • Provable Dynamic Fusion for Low-Quality Multimodal Data cites Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on22

  1. Probabilistic Face EmbeddingsYichun Shi, Anil K. JainICCV 2019 · 362 citations
  2. Balanced Multimodal Learning via On-the-fly Gradient ModulationXiaokang Peng, Yake Wei, Andong Deng, Dong Wang et al.CVPR 2022 · 264 citations
  3. ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic SegmentationJinming Cao, Hanchao Leng, Dani Lischinski, Danny Cohen-Or et al.ICCV 2021 · 186 citations
  4. Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang et al.ICML 2022 · 168 citations
  5. Provable Dynamic Fusion for Low-Quality Multimodal DataQingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu et al.ICML 2023 · 143 citations

Related papers

  1. CMoB: Modality Valuation via Causal Effect for Balanced Multimodal LearningJun Wang, Fuyuan Cao, Zhixin Xue, Xingwang Zhao et al.NeurIPS 2025 · 4 citations
  2. Bootstrapping Multi-view Learning for Test-time Noisy CorrespondenceChanghao He, Di Xue, Shuxian Li, Yanji Hao et al.CVPR 2026
  3. A Theoretical Proof of Dynamic Multimodal Fusion Exacerbates Modality GreedyXiaorui Ding, Huan Ma, Changqing ZhangACM MM 2025
  4. GMML: Gradient-Modulated Robustness for Imbalance-Aware Multimodal LearningZikai Zhang, Xu Zhang, Ziyi Li, Yidong Li et al.ACM MM 2025 · 1 citation
  5. Calibrating Multimodal LearningHuan Ma, Qingyang Zhang, Changqing Zhang, Bingzhe Wu et al.ICML 2023 · 42 citations
Dusk over the sea between two cliffs drawn in fine vertical lines