Lune

EMNLP Findings2025

The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems

Chanwoo Choi, Jinsoo Kim, Sukmin Cho, Soyeong Jeong, Buru Chang

Abstract

With the growing adoption of retrievalaugmented generation (RAG) systems, various attack methods have been proposed to degrade their performance. However, most existing approaches rely on unrealistic assumptions in which external attackers have access to internal components such as the retriever. To address this issue, we introduce a realistic black-box attack based on the RAG paradox, a structural vulnerability arising from the system's effort to enhance trust by revealing both the retrieved documents and their sources to users. This transparency enables attackers to observe which sources are used and how information is phrased, allowing them to craft poisoned documents that are more likely to be retrieved and upload them to the identified sources. Moreover, as RAG systems directly provide retrieved content to users, these documents must not only be retrievable but also appear natural and credible to maintain user confidence in the search results. Unlike prior work that focuses solely on improving document retrievability, our attack method explicitly considers both retrievability and user trust in the retrieved content. Both offline and online experiments demonstrate that our method significantly degrades system performance without internal access, while generating natural-looking poisoned documents.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Inference Cost Attacks for Retrieval-Augmented Large Language Models cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Reranker Helps, but Not Enough: Towards Strong Poisoning Attacks Against Retrieval-Augmented Generation cites Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations
  • Reranker Helps, but Not Enough: Towards Strong Poisoning Attacks Against Retrieval-Augmented Generation cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • Reranker Helps, but Not Enough: Towards Strong Poisoning Attacks Against Retrieval-Augmented Generation cites AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
  • Reranker Helps, but Not Enough: Towards Strong Poisoning Attacks Against Retrieval-Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks cites Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations
  • MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks cites AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
  • MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning AttacksTailun Chen, Yu He, Yan Wang, Shuo Shao et al.CCS 2026 · 4 citations
  2. Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAGSarthak Choudhary, Nils Palumbo, Ashish Hooda, Krishnamurthy Dvijotham et al.ICML 2026 · 3 citations
  3. Inference Cost Attacks for Retrieval-Augmented Large Language ModelsChengliang Liu, Liangbo Ning, Yujuan Ding, Wenqi FanWWW 2026
  4. Reranker Helps, but Not Enough: Towards Strong Poisoning Attacks Against Retrieval-Augmented GenerationXiaokun Yang, Jian Liang, Yesheng Liu, Xin Xiong et al.ICML 2026

Builds on10

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  3. AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge BasesZhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song et al.NeurIPS 2024 · 539 citations
  4. Text Embeddings Reveal (Almost) As Much As TextJohn X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. RushEMNLP 2023 · 60 citations
  5. Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level PerturbationsSukmin Cho, Soyeong Jeong, Jeongyeon Seo, Taeho Hwang et al.EMNLP Findings 2024 · 14 citations

Related papers

  1. RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement LearningMeng Xi, Sihan Lv, Yechen Jin, Guanjie Cheng et al.ACL Findings 2026
  2. FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation ModelsZhuo Chen, Yuyang Gong, Jiawei Liu, Miaokun Chen et al.CCS 2025
  3. WARP: A Word-Level Backdoor Attack Targeting RAG Systems via Retrieval Corpus PoisoningHui Liu, Yibo Zhou, Liguo Dong, Weidong Li et al.KDD 2026
  4. GraphRAG Under FireJiacheng Liang, Yuhui Wang, Changjiang Li, Tanqiu Jiang et al.S&P 2026 · 31 citations
  5. Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation SystemsHaowei Wang, Rupeng Zhang, Junjie Wang, Mingyang Li et al.AAAI 2026 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines