Lune

ACL Findings2026

Training-Free Adaptive Speculative Decoding via Linguistic Priors

Jingyi Wang, Jiaqi Huang, Zunnan Xu, Jun Zhou, Kehong Yuan, Xiang Qian

Abstract

Speculative decoding (SPD) has emerged as a promising technique to accelerate Large Language Model (LLM) inference. However, current approaches typically enforce a uniform verification standard, neglecting the inherent heterogeneity of natural language and failing to distinguish between semantically-rich content and structurally-predictable syntax. In this paper, we propose LinguaSpec, a training-free framework that leverages linguistic priors to enable adaptive drafting and verification. Specifically, we introduce: (1) a Static Linguistic Probe (SLP) to categorize tokens with zero latency; (2) Syntactic Normalized Surprisal (SNS) to calibrate uncertainty against categoryspecific entropy; and (3) a dual strategy of Syntactically-Guided Elastic Expansion and POS-Adaptive Deferred Verification to dynamically adjust drafting depth and verification rigor. By balancing semantic integrity with structural efficiency, LinguaSpec significantly accelerates inference without requiring additional training. Experimental results demonstrate its superior performance across diverse benchmarks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Break the Sequential Dependency of LLM Inference Using Lookahead Decoding cites Fast Inference from Transformers via Speculative Decoding
  • EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test cites Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  • EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test cites EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
  • EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test cites Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
  • EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test cites Fast Inference from Transformers via Speculative Decoding
  • EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty cites Fast Inference from Transformers via Speculative Decoding
  • Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads cites Fast Inference from Transformers via Speculative Decoding
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasICML 2023 · 1,472 citations
  2. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding HeadsTianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng et al.ICML 2024 · 669 citations
  3. EAGLE: Speculative Sampling Requires Rethinking Feature UncertaintyYuhui Li, Fangyun Wei, Chao Zhang, Hongyang ZhangICML 2024 · 424 citations
  4. EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time TestYuhui Li, Fangyun Wei, Chao Zhang, Hongyang ZhangNeurIPS 2025 · 347 citations
  5. Break the Sequential Dependency of LLM Inference Using Lookahead DecodingYichao Fu, Peter Bailis, Ion Stoica, Hao ZhangICML 2024 · 290 citations

Related papers

  1. UniSpec: Training-Free Speculative Decoding for Robust LLM Acceleration Across Languages and HardwareTruong Dinh Do, Nguyen-Khang Le, Le-Minh NguyenACL 2026
  2. RACER: Retrieval-Augmented Contextual Rapid Speculative DecodingZihong Zhang, Zuchao Li, Lefei Zhang, Ping Wang et al.ACL Findings 2026 · 1 citation
  3. Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact MatchJinze Li, Yixing Xu, Guanchen Li, Shuo Yang et al.ICLR 2026 · 12 citations
  4. LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token SpeculationTianyu Liu, Qitan Lv, Hao Li, Xing Gao et al.ACL Findings 2026
  5. Block Verification Accelerates Speculative DecodingZiteng Sun, Uri Mendlovic, Yaniv Leviathan, Asaf Aharoni et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines