Lune

EMNLP Findings2024

Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework

Lu Chen, Ruqing Zhang, Jiafeng Guo, Yixing Fan, Xueqi Cheng

Abstract

Retrieval-augmented generation (RAG) has emerged as a popular solution to mitigate the hallucination issues of large language models. However, existing studies on RAG seldom address the issue of predictive uncertainty, i.e., how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. In this work, we emphasize the importance of risk control, ensuring that RAG models proactively refuse to answer questions with low confidence. Our research identifies two critical latent factors affecting RAG's confidence in its predictions: the quality of the retrieved results and the manner in which these results are utilized. To guide RAG models in assessing their own confidence based on these two latent factors, we develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers. We also introduce a benchmarking procedure to collect answers with the option to abstain, facilitating a series of experiments. For evaluation, we introduce several risk-related metrics and the experimental results demonstrate the effectiveness of our approach. Our code and benchmark dataset are available at https://github.com/ict-bigdatalab/RC-RAG.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Improving Language Models by Retrieving from Trillions of Tokens cites Language Models are Few-Shot Learners
  • Improving Language Models by Retrieving from Trillions of Tokens cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Large Language Models Struggle to Learn Long-Tail Knowledge cites Language Models are Few-Shot Learners
  • Large Language Models Struggle to Learn Long-Tail Knowledge cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting LayersYoumin Ko, Sungjong Seo, Hyunjoon KimNeurIPS 2025 · 2 citations
  2. When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented GenerationJing Ren, Bowen Li, Ziqi Xu, Xikun Zhang et al.WWW 2026

Builds on26

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  3. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
  4. Improving Language Models by Retrieving from Trillions of TokensSebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai et al.ICML 2022 · 1,629 citations
  5. Large Language Models Struggle to Learn Long-Tail KnowledgeNikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace et al.ICML 2023 · 623 citations

Related papers

  1. CtrlA: Adaptive Retrieval-Augmented Generation via Inherent ControlHuanshuo Liu, Hao Zhang, Zhijiang Guo, Jing Wang et al.ACL Findings 2025 · 2 citations
  2. Benchmarking Large Language Models in Retrieval-Augmented GenerationJiawei Chen, Hongyu Lin, Xianpei Han, Le SunAAAI 2024 · 531 citations
  3. Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented GenerationEkaterina Fadeeva, Aleksandr Rubashevskii, Dzianis Piatrashyn, Roman Vashurin et al.ACL Findings 2026 · 2 citations
  4. MultiRAG: A Knowledge-Guided Framework for Mitigating Hallucination in Multi-Source Retrieval Augmented GenerationWenlong Wu, Haofen Wang, Bohan Li, Peixuan Huang et al.ICDE 2025 · 16 citations
  5. C-RAG: Certified Generation Risks for Retrieval-Augmented Language ModelsMintong Kang, Nezihe Merve Gürel, Ning Yu, Dawn Song et al.ICML 2024 · 33 citations
Dusk over the sea between two cliffs drawn in fine vertical lines