MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification
Yujian Zhao, Hankun Liu, Guanglin Niu
Abstract
Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery has recently emerged as a critical yet underexplored task in maritime intelligence and surveillance. However, the substantial modality gap between optical and SAR images poses a major challenge for robust identification. To address this issue, we propose MOS, a novel framework designed to mitigate the optical-SAR modality gap and achieve modality-consistent feature learning for optical-SAR cross-modal ship ReID. MOS consists of two core components: (1) Modality-Consistent Representation Learning (MCRL) applies denoise SAR image procession and a class-wise modality alignment loss to align intra-identity feature distributions across modalities. (2) Cross-modal Data Generation and Feature fusion (CDGF) leverages a brownian bridge diffusion model to synthesize cross-modal samples, which are subsequently fused with original features during inference to enhance alignment and discriminability. Extensive experiments on the HOSS ReID dataset demonstrate that MOS significantly surpasses state-of-the-art methods across all evaluation protocols, achieving notable improvements of +3.0%, +6.2%, and +16.4% in R1 accuracy under the ALL to ALL, Optical to SAR, and SAR to Optical settings, respectively. The code and trained models will be released upon publication.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 0a97ada2-83d5-4c99-b1e7-8fb55ba35ba8Builds on19
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations
- Training data-efficient image transformers & distillation through attentionHugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa et al.ICML 2021 · 8,974 citations
Related papers
- Cross-Modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and MethodHan Wang, Shengyang Li, Jian Yang, Yuxuan Liu et al.ICCV 2025 · 10 citations
- Joint Color-irrelevant Consistency Learning and Identity-aware Modality Adaptation for Visible-infrared Cross Modality Person Re-identificationZhiwei Zhao, Bin Liu, Qi Chu, Yan Lu et al.AAAI 2021 · 92 citations
- Interpretable Matching of Optical-SAR Image via Dynamically Conditioned Diffusion ModelsShuiping Gou, Xin Wang, Xinlin Wang, Yunzhi ChenACM MM 2024 · 1 citation
- Syncretic Modality Collaborative Learning for Visible Infrared Person Re-IdentificationZiyu Wei, Xi Yang, Nannan Wang, Xinbo GaoICCV 2021 · 173 citations
- Infrared-Visible Cross-Modal Person Re-Identification with an X ModalityDiangang Li, Xing Wei, Xiaopeng Hong, Yihong GongAAAI 2020 · 419 citations
