ACL Findings2026
X-Router: Decoupling Knowledge and Reasoning for Cost-Effective LLM Inference
Zixuan Wang, Yinze Ding, Zihan Wang, Jinyu Guo, Zhenhong Zhou, Junhao Dong, Chaomeng Chen
Abstract
Large Language Models (LLMs) are often augmented with Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) prompting, yet static "always-on" use is computationally wasteful. Existing adaptive methods typically optimize a single axis, overlooking that evidence need and reasoning depth are only partially correlated. We present X-ROUTER, a dual-axis routing framework that separates retrieval necessity from reasoning necessity under a user-defined cost-quality trade-off. Offline, X-ROUTER profiles four pipelines (DIRECT, RAG, COT, RAG+COT) and derives supervision by selecting the utilitymaximizing strategy that trades answer quality against token usage and latency. Online, a compact dual-head router, conditioned on cost weights, uses lightweight probesretrieval-score dispersion (NQC) and singlepass draft negative log-likelihood (NLL)-to decide whether to invoke RAG and/or CoT without sampling or model internals. Across six QA benchmarks, X-ROUTER reduces token usage by up to 86% and latency by up to 84% while improving answer quality over strong baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Large Language Models are Zero-Shot Reasoners
- Large Language Models are Zero-Shot Reasoners cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds on21
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
Related papers
- Think When Needed: Model-Aware Reasoning Routing for LLM-based RankingHuizhong Guo, Tianjun Wei, Dongxia Wang, Yingpeng Du et al.SIGIR 2026
- RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts ReasoningYuhang Liu, Ruijie Wang, Yunlong Chu, Bing Hao et al.KDD 2026 · 1 citation
- SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved ContextHairu Wang, Yuan Feng, Yukun Cao, Xike Xie et al.EMNLP Findings 2025 · 1 citation
- RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement LearningYucan Guo, Miao Su, Saiping Guan, Zihao Sun et al.ACL Findings 2026 · 2 citations
- Intention Chain-of-Thought Prompting with Dynamic Routing for Code GenerationShen Li, Li Huang, Shaoxiong Zhan, Weifeng Sun et al.AAAI 2026 · 1 citation
