Lune

EMNLP Findings2025

Adaptive LLM Routing under Budget Constraints

Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, Sho Takemori, Vishal Sharma

Abstract

Large Language Models (LLMs) have revolutionized natural language processing, but their varying capabilities and costs pose challenges in practical applications. LLM routing addresses this by dynamically selecting the most suitable LLM for each query/task. Previous approaches treat this as a supervised learning problem, assuming complete knowledge of optimal query-LLM pairings. However, realworld scenarios lack such comprehensive mappings and face evolving user queries. We thus propose to study LLM routing as a contextual bandit problem, enabling adaptive decisionmaking using bandit feedback without requiring exhaustive inference across all LLMs for all queries (in contrast to supervised routing). To address this problem, we develop a shared embedding space for queries and LLMs, where query and LLM embeddings are aligned to reflect their affinity. This space is initially learned from offline human preference data and refined through online bandit feedback. We instantiate this idea through Preference-prior Informed LinUCB fOr Adaptive RouTing (PILOT), a novel extension of LinUCB. To handle diverse user budgets for model routing, we introduce an online cost policy modeled as a multi-choice knapsack problem, ensuring resource-efficient routing.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing cites LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure PerspectiveJiin Kim, Byeongjun Shin, Jinha Chung, Minsoo RhuHPCA 2026 · 7 citations

Builds on8

  1. Chatbot Arena: An Open Platform for Evaluating LLMs by Human PreferenceWei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos et al.ICML 2024 · 1,212 citations
  2. Hybrid LLM: Cost-Efficient and Quality-Aware Query RoutingDujian Ding, Ankur Mallick, Chi Wang, Robert Sim et al.ICLR 2024 · 282 citations
  3. Simple linear attention language models balance the recall-throughput tradeoffSimran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina et al.ICML 2024 · 154 citations
  4. One Embedder, Any Task: Instruction-Finetuned Text EmbeddingsHongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang et al.ACL Findings 2023 · 134 citations
  5. LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative FusionDongfu Jiang, Xiang Ren, Bill Yuchen LinACL 2023 · 95 citations

Related papers

  1. Online Multi-LLM Selection via Contextual Bandits Under Unstructured Context EvolutionManhin Poon, Xiangxiang Dai, Xutong Liu, Fang Kong et al.AAAI 2026 · 11 citations
  2. IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response TheoryWei Song, Zhenya Huang, Cheng Cheng, Weibo Gao et al.ACL 2025 · 20 citations
  3. BEST-Route: Adaptive LLM Routing with Test-Time Optimal ComputeDujian Ding, Ankur Mallick, Shaokun Zhang, Chi Wang et al.ICML 2025
  4. BARouter: A Budget-adaptive Online Large Language Model Router FrameworkLingkai Zu, Xiyue Peng, Xin LiuWWW 2026
  5. ICL-Router: In-Context Learned Model Representations for LLM RoutingChenxu Wang, Hao Li, Yiqun Zhang, Linyao Chen et al.AAAI 2026 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines