EMNLP Findings2025
RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis
Xue Tan, Hao Luan, Mingyu Luo, Xiaoyan Sun, Ping Chen, Jun Dai
Abstract
Retrieval-Augmented Generation (RAG) enriches the input to LLMs by retrieving information from the relevant knowledge database, enabling them to produce responses that are more accurate and contextually appropriate.It is worth noting that the knowledge database, being sourced from publicly available channels such as Wikipedia, inevitably introduces a new attack surface.RAG poisoning attack involves injecting malicious texts into the knowledge database, ultimately leading to the generation of the attacker's target response (also called poisoned response).However, there are currently limited methods available for detecting such poisoning attacks.We aim to bridge the gap in this work by introducing RevPRAG, a flexible and automated detection pipeline that leverages the activations of LLMs for poisoned response detection.Our investigation uncovers distinct patterns in LLMs' activations when generating poisoned responses versus correct responses.Our results on multiple benchmarks and RAG architectures show our approach can achieve a 98% true positive rate, while maintaining a false positive rate close to 1%.Query Answer LLM Prompt Context: [...] Everest stands the tallest [...
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation cites Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context cites Measuring and Narrowing the Compositionality Gap in Language Models
Cited by top-tier papers5
- SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGXiaonan Si, Meilin Zhu, Simeng Qin, Lijia Yu et al.NeurIPS 2025 · 16 citations
- Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM SafetySeongmin Lee, Aeree Cho, Grace C. Kim, Shengyun Peng et al.EMNLP 2025 · 1 citation
- Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation DetectionZehong Yan, Peng Qi, Wynne Hsu, Mong-Li LeeICDE 2026 · 1 citation
- InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented GenerationJiachang Zhang, Min Chen, Xiao Ren, Zhenyong Zhang et al.CCS 2026
- One Single Hub Text Breaks CLIP: Identifying Vulnerabilities in Cross-Modal Encoders via HubnessHiroyuki Deguchi, Katsuki Chousa, Yusuke SakaiACL 2026
Builds on15
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
- On Adaptive Attacks to Adversarial Example DefensesFlorian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander MadryNeurIPS 2020 · 1,026 citations
- Making Retrieval-Augmented Language Models Robust to Irrelevant ContextOri Yoran, Tomer Wolfson, Ori Ram, Jonathan BerantICLR 2024 · 361 citations
- Measuring and Narrowing the Compositionality Gap in Language ModelsOfir Press, Muru Zhang, Sewon Min, Ludwig Schmidt et al.EMNLP Findings 2023 · 296 citations
Related papers
- On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application DomainsXun Xian, Ganghua Wang, Xuan Bi, Rui Zhang et al.ICML 2025
- RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement LearningMeng Xi, Sihan Lv, Yechen Jin, Guanjie Cheng et al.ACL Findings 2026
- KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented GenerationQizhi Chen, Chao Qi, Yihong Huang, Muquan Li et al.WWW 2026
- PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel OptimizationYang Jiao, Xiaodong Wang, Kai YangSIGIR 2025 · 6 citations
- Traceback of Poisoning Attacks to Retrieval-Augmented GenerationBaolei Zhang, Haoran Xin, Minghong Fang, Zhuqing Liu et al.WWW 2025 · 20 citations
