Rethinking Multi-Modal Object Detection From the Perspective of Mono-Modality Feature Learning

Multi-Modal Object Detection (MMOD), due to its stronger adaptability to various complex environments, has been widely applied in various applications. Extensive research is dedicated to the RGB-IR object detection, primarily focusing on how to integrate complementary features from RGB-IR modalities. However, they neglect the monomodality insufficient learning problem, which arises from decreased feature extraction capability in multi-modal joint learning. This leads to a prevalent but unreasonable phe-nomenon-Fusion Degradation, which hinders the performance improvement of the MMOD model. Motivated by this, in this paper, we introduce linear probing evaluation to the multi-modal detectors and rethink the multimodal object detection task from the mono-modality learning perspective. Therefore, we construct a novel framework called $\boldsymbol{M}^{2} \boldsymbol{D}$-LIF, which consists of the Mono-Modality Distillation ($\boldsymbol{M}^{2} \boldsymbol{D}$) method and the Local Illuminationaware Fusion (LIF) module. The $M^{2}$ D-LIF framework facilitates the sufficient learning of mono-modality during multi-modal joint training and explores a lightweight yet effective feature fusion manner to achieve superior object detection performance. Extensive experiments conducted on three MMOD datasets demonstrate that our $M^{2}$ D-LIF effectively mitigates the Fusion Degradation phenomenon and outperforms the previous SOTA detectors. The codes are available at https://github.com/Zhao-Tian-yi/M2D-LIF.