MMSD3.0: A Multi-Image Benchmark for Real-World Multimodal Sarcasm Detection
Haochen Zhao, Yuyao Kong, Yongxiu Xu, Gaopeng Gou, Hongbo Xu, Yubin Wang, Haoliang Zhang
摘要
Despite progress in multimodal sarcasm detection, existing datasets and methods predominantly focus on single-image scenarios, overlooking potential semantic and affective relations across multiple images. This leaves a gap in modeling cases where sarcasm is triggered by multi-image cues in real-world settings. To bridge this gap, we introduce MMSD3.0, a new benchmark composed entirely of multi-image samples curated from tweets and Amazon reviews. We further propose the Cross-Image Reasoning Model (CIRM), which performs targeted cross-image sequence modeling to capture latent inter-image connections. In addition, we introduce a relevance-guided, fine-grained cross-modal fusion mechanism based on text-image correspondence to reduce information loss during integration. We establish a comprehensive suite of strong and representative baselines and conduct extensive experiments, showing that MMSD3.0 is an effective and reliable benchmark that better reflects real-world conditions. Moreover, CIRM demonstrates state-of-the-art performance across MMSD, MMSD2.0 and MMSD3.0, validating its effectiveness in both single-image and multi-image scenarios. Dataset and code are publicly available at https://github.com/ZHCMOONWIND/MMSD3.0.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- Reasoning with Multimodal Sarcastic Tweets via Modeling Cross-Modality Contrast and Semantic AssociationNan Xu, Zhixiong Zeng, Wenji MaoACL 2020 · 被引用 153 次
- Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge EnhancementHui Liu, Wenya Wang, Haoliang LiEMNLP 2022 · 被引用 91 次
- Mutual-Enhanced Incongruity Learning Network for Multi-Modal Sarcasm DetectionYang Qiao, Liqiang Jing, Xuemeng Song, Xiaolin Chen 等AAAI 2023 · 被引用 84 次
- MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm DetectionYifeng Xie, Zhihong Zhu, Xin Chen, Zhanpeng Chen 等ACM MM 2024 · 被引用 11 次
相关 Paper
- Multi-Modal Sarcasm Detection with Interactive In-Modal and Cross-Modal GraphsBin Liang, Chenwei Lou, Xiang Li, Lin Gui 等ACM MM 2021 · 被引用 128 次
- DocMSU: A Comprehensive Benchmark for Document-Level Multimodal Sarcasm UnderstandingHang Du, Guoshun Nan, Sicheng Zhang, Binzhu Xie 等AAAI 2024 · 被引用 9 次
- Well, Now We Know! Unveiling Sarcasm: Initiating and Exploring Multimodal Conversations with ReasoningGopendra Vikram Singh, Mauajama Firdaus, Dushyant Singh Chauhan, Asif Ekbal 等AAAI 2024 · 被引用 6 次
- DIP: Dual Incongruity Perceiving Network for Sarcasm DetectionChangsong Wen, Guoli Jia, Jufeng YangCVPR 2023
- Multimodal Sarcasm Target Identification in TweetsJiquan Wang, Lin Sun, Yi Liu, Meizhi Shao 等ACL 2022 · 被引用 28 次
