Task-Gated Multi-Expert Collaboration Network for Degraded Multi-Modal Image Fusion
Yiming Sun, Xin Li, Pengfei Zhu, Qinghua Hu, Dongwei Ren, Huiying Xu, Xinzhong Zhu
Abstract
Multi-modal image fusion aims to integrate complementary information from different modalities to enhance perceptual capabilities in applications such as rescue and security. However, real-world imaging often suffers from degradation issues, such as noise, blur, and haze in visible imaging, as well as stripe noise in infrared imaging, which significantly degrades model performance. To address these challenges, we propose a task-gated multi-expert collaboration network (TG-ECNet) for degraded multi-modal image fusion. The core of our model lies in the task-aware gating and multi-expert collaborative framework, where the task-aware gating operates in two stages: degradation-aware gating dynamically allocates expert groups for restoration based on degradation types, and fusion-aware gating guides feature integration across modalities to balance information retention between fusion and restoration tasks. To achieve this, we design a two-stage training strategy that unifies the learning of restoration and fusion tasks. This strategy resolves the inherent conflict in information processing between the two tasks, enabling all-in-one multi-modal image restoration and fusion. Experimental results demonstrate that TG-ECNet significantly enhances fusion performance under diverse complex degradation conditions and improves robustness in downstream applications. The code is available at https: //github.com/LeeX54946/TG-ECNet.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- CtrlFuse: Mask-Prompt Guided Controllable Infrared and Visible Image Fusion cites Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection
- Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion cites Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection
- PromptIR: Prompting for All-in-One Image Restoration cites All-In-One Image Restoration for Unknown Corruption
- DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion cites Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection
- DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion cites Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity
Cited by top-tier papers4
- Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image FusionZengyi Yang, Yu Liu, Juan Cheng, Zhiqin Zhu et al.CVPR 2026 · 7 citations
- CtrlFuse: Mask-Prompt Guided Controllable Infrared and Visible Image FusionYiming Sun, Yuan Ruan, Qinghua Hu, Pengfei ZhuAAAI 2026
- MMVIP: A Visible-infrared Paired Dataset for Multi-weather Marine VisionYunpeng Yin, Lihan Wang, Zhaoshen He, Xinqiang He et al.CVPR 2026
- DroneDINO: Towards Heterogeneous Routed Mixture of Experts for Drone-based Unified Object DetectionRui Chen, Dongdong Li, Yan Fan, Yan Liu et al.ICML 2026
Builds on14
- Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object DetectionJinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu et al.CVPR 2022 · 929 citations
- Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and IntensityHao Zhang, Han Xu, Yang Xiao, Xiaojie Guo et al.AAAI 2020 · 583 citations
- PromptIR: Prompting for All-in-One Image RestorationVaishnav Potlapalli, Syed Waqas Zamir, Salman H. Khan, Fahad Shahbaz KhanNeurIPS 2023 · 386 citations
- DDFM: Denoising Diffusion Model for Multi-Modality Image FusionZixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang et al.ICCV 2023 · 350 citations
- All-In-One Image Restoration for Unknown CorruptionBoyun Li, Xiao Liu, Peng Hu, Zhongqin Wu et al.CVPR 2022 · 338 citations
Related papers
- ALLNet: Multi-task Dense Prediction for Degraded ImagesWeiran Wang, Jialing Wu, Yaqi Chang, Gang He et al.CVPR 2026
- Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image FusionBing Cao, Yiming Sun, Pengfei Zhu, Qinghua HuICCV 2023 · 110 citations
- Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image FusionXunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang et al.CVPR 2024 · 121 citations
- MdaIF: Robust One-Stop Multi-Degradation-Aware Image Fusion with Language-Driven SemanticsJing Li, Yifan Wang, Jiafeng Yan, Renlong Zhang et al.AAAI 2026
- AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and ModulationYuning Cui, Syed Waqas Zamir, Salman H. Khan, Alois Knoll et al.ICLR 2025 · 9 citations
