Cognitive Distillation for Information Forensics: Towards Improved Hateful Meme Detection

Internet meme is a mainstream multimodal content form that often conveys users' viewpoints metaphorically and may also carry implicit hateful information, making hateful meme detection an extremely challenging task. Recent evidence-enhanced methods achieve remarkable research progress by retrieving meme-related evidence samples for analogical reasoning. However, most existing methods rely solely on image-text semantic similarity for information forensics, ignoring the key characteristic of meme that their meanings are shaped by the collective cognition of diverse users. To address this issue, we propose a Cognition-driven Information Forensics (CIF) framework for hateful meme detection, which integrates the collective cognition of diverse users into the entire process of information forensics and hateful content detection. This framework consists of three core modules: (1) Diverse User Comments Acquisition: it adopts a multi-agent comment simulation system to generate diverse user comments for capturing the characteristics of collective cognition; (2) Feature to Cognition Alignment Distillation: it maps visual and textual features to a semantic space aligned with collective cognition; (3) Dual-Level Association Learning: it strengthens the mining of implicit hateful clues by modeling the intra-sample and inter-sample correlation relationships. Experimental results demonstrate that the CIF framework significantly improves detection performance on mainstream datasets. In particular, on the most challenging FHM dataset, its detection performance is 4.2% higher than that of the current state-of-the-art model. Caution: Contains academic discussions of hatespeech; viewer discretion advised.