SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding
Thomas Walton, Darin Tsui, Aryan Musharaf, Amirali Aghazadeh
Abstract
Autoregressive models have transformed protein engineering by enabling the generation of novel protein sequences beyond those found in nature. However, their sequential inference introduces significant latency, limiting their utility in high-throughput protein screening. Speculative decoding accelerates generation by employing a lightweight draft model to sample tokens, which a larger target model then verifies and refines. Yet, in protein sequence generation, draft models are typically agnostic to the structural and functional constraints of the target protein, leading to biologically implausible outputs and a shift in the likelihood distribution of generated sequences. We introduce SpecMER (Speculative Decoding via k-mer Guidance), a novel framework that incorporates biological, structural, and functional priors using k-mer motifs extracted from multiple sequence alignments. By scoring candidate sequences in parallel and selecting those most consistent with known biological patterns, SpecMER significantly improves sequence plausibility while retaining the efficiency of speculative decoding. SpecMER achieves 24-32% speedup over standard autoregressive decoding, along with higher acceptance rates and improved sequence likelihoods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads cites Fast Inference from Transformers via Speculative Decoding
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads cites The Curious Case of Neural Text Degeneration
Builds on8
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasICML 2023 · 1,472 citations
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding HeadsTianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng et al.ICML 2024 · 669 citations
- Learning inverse folding from millions of predicted structuresChloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin et al.ICML 2022 · 560 citations
- Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time RetrievalPascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado et al.ICML 2022 · 236 citations
Related papers
- Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq GenerationHeming Xia, Tao Ge, Peiyi Wang, Si-Qing Chen et al.EMNLP Findings 2023 · 40 citations
- DFlash: Block Diffusion for Flash Speculative DecodingJian Chen, Yesheng Liang, Zhijian LiuICML 2026
- LK Losses: Direct Acceptance Rate Optimization for Speculative DecodingAlexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skvortsov et al.ICML 2026 · 11 citations
- Traversal Verification for Speculative Tree DecodingYepeng Weng, Qiao Hu, Xujie Chen, Li Liu et al.NeurIPS 2025 · 11 citations
- Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Auto SpeculationHengyuan Hu, Aniket Das, Dorsa Sadigh, Nima AnariICML 2025
