MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification
Yujian Zhao, Hankun Liu, Guanglin Niu
摘要
Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery has recently emerged as a critical yet underexplored task in maritime intelligence and surveillance. However, the substantial modality gap between optical and SAR images poses a major challenge for robust identification. To address this issue, we propose MOS, a novel framework designed to mitigate the optical-SAR modality gap and achieve modality-consistent feature learning for optical-SAR cross-modal ship ReID. MOS consists of two core components: (1) Modality-Consistent Representation Learning (MCRL) applies denoise SAR image procession and a class-wise modality alignment loss to align intra-identity feature distributions across modalities. (2) Cross-modal Data Generation and Feature fusion (CDGF) leverages a brownian bridge diffusion model to synthesize cross-modal samples, which are subsequently fused with original features during inference to enhance alignment and discriminability. Extensive experiments on the HOSS ReID dataset demonstrate that MOS significantly surpasses state-of-the-art methods across all evaluation protocols, achieving notable improvements of +3.0%, +6.2%, and +16.4% in R1 accuracy under the ALL to ALL, Optical to SAR, and SAR to Optical settings, respectively. The code and trained models will be released upon publication.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper19
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 被引用 11,724 次
- Training data-efficient image transformers & distillation through attentionHugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa 等ICML 2021 · 被引用 8,974 次
相关 Paper
- Cross-Modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and MethodHan Wang, Shengyang Li, Jian Yang, Yuxuan Liu 等ICCV 2025 · 被引用 10 次
- Joint Color-irrelevant Consistency Learning and Identity-aware Modality Adaptation for Visible-infrared Cross Modality Person Re-identificationZhiwei Zhao, Bin Liu, Qi Chu, Yan Lu 等AAAI 2021 · 被引用 92 次
- Interpretable Matching of Optical-SAR Image via Dynamically Conditioned Diffusion ModelsShuiping Gou, Xin Wang, Xinlin Wang, Yunzhi ChenACM MM 2024 · 被引用 1 次
- Syncretic Modality Collaborative Learning for Visible Infrared Person Re-IdentificationZiyu Wei, Xi Yang, Nannan Wang, Xinbo GaoICCV 2021 · 被引用 173 次
- Infrared-Visible Cross-Modal Person Re-Identification with an X ModalityDiangang Li, Xing Wei, Xiaopeng Hong, Yihong GongAAAI 2020 · 被引用 419 次
