From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework
Yufei Chen, Yao Wang, Haibin Zhang, Hualin zhou, Tao Gu
Abstract
Retrieval-augmented generation (RAG) systems suffer from a fundamental functional misalignment where retrievers optimize for semantic relevance, often recalling documents with high background utility but factually erroneous answer spans that generators blindly adopt as cognitive shortcuts. To resolve this, we propose the collaborative Critic-Reasoner framework that shifts robustness control from coarse-grained filtering to fine-grained cognitive decoupling. We disentangle the generation process into two serialized roles by deploying a Critic to perform surgical evidence purification through identifying and masking misleading entities while preserving supportive background context, followed by a Reasoner that switches from rote extraction to deductive reasoning based on the residual evidence. We operationalize this framework via a two-stage alignment strategy combining supervised fine-tuning (SFT) with path-aware direct preference optimization (DPO) to enforce strict behavioral synergy. Experimental results on adversarial benchmarks such as ConFiQA demonstrate that our method significantly outperforms baselines, achieving a 25.99% accuracy gain in conflicting scenarios and effectively resolving the trust bias dilemma in real-world RAG.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on12
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationJianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo et al.ACL Findings 2024 · 762 citations
- Large Language Models Struggle to Learn Long-Tail KnowledgeNikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace et al.ICML 2023 · 623 citations
- Retrieval Augmentation Reduces Hallucination in ConversationKurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela et al.EMNLP Findings 2021 · 579 citations
Related papers
- Beyond Semantic Relevance: Counterfactual Risk Minimization for Robust Retrieval-Augmented GenerationPeiyang Liu, Qiang Yan, Ziqiang Cui, Di Liang et al.SIGIR 2026
- Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language ModelsCan Xu, Lingyong Yan, Jiayi Wu, Haosen Wang et al.ACL Findings 2026
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and OptimizationJiaqi Wei, Hao Zhou, Xiang Zhang, Di Zhang et al.NeurIPS 2025 · 14 citations
- Conflict-Aware RAG: Multi-Stage Learning with Conflict Signals for Robust Retrieval-Augmented GenerationHaiyan Wu, Chenchen Wang, Chaoqun Sun, Chengxiong Lu et al.WWW 2026
- Counterfactual Reasoning for Retrieval-Augmented GenerationHuaiyu Qin, Chunyu Wei, Yueguo Chen, Yunhai WangICLR 2026
