RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning

Retrieval-Augmented Generation (RAG) systems based on Large Language Models (LLMs) have become a core technology for tasks such as question-answering (QA) and content generation. RAG poisoning is an attack method to induce LLMs to generate the attacker's expected text by injecting poisoned documents into the database of RAG systems. Existing research can be broadly divided into two classes: white-box methods and black-box methods. White-box methods utilize gradient information to optimize poisoned documents, and black-box methods use a pre-trained LLM to generate them. However, existing white-box methods require knowledge of the RAG system's internal composition and implementation details, whereas black-box methods are unable to utilize interactive information. In this work, we propose the RIPRAG attack framework, an end-to-end attack pipeline that treats the target RAG system as a black box and leverages our proposed Reinforcement Learning from Black-box Feedback (RLBF) method to optimize the generation model for poisoned documents. We designed two kinds of rewards: similarity reward and attack reward. Experimental results demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems, achieving an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods. This highlights prevalent deficiencies in current defensive methods and provides critical insights for LLM security research. Introduction RAG [1] has been proposed to mitigate the inherent limitation of LLMs, which lies in the static nature of their parametric knowledge that can become outdated or lack specificity for certain domains. By equipping LLMs with access to an external, updatable database, this paradigm enhances the factuality and relevance of generated responses, particularly in critical applications like question-answering and content generation, through dynamic retrieval and grounding of responses in pertinent information. Despite its advantages, the RAG framework introduces new vulnerabilities, primarily through its retrieval component. A significant threat is RAG poisoning [2] , where attackers inject poisoned documents into the database to manipulate † Meng Xi and Sihan Lv contributed equally to this work. * Guanjie Cheng is the corresponding author.