Speculative Contrastive Decoding

Large language models (LLMs) exhibit ex-001 ceptional performance in language tasks, yet 002 their auto-regressive inference is limited due to 003 high computational requirements and is sub-004 optimal due to the exposure bias. Inspired 005 by speculative decoding and contrastive de-006 coding, we introduce Speculative Contrastive 007 Decoding (SCD), a straightforward yet pow-008 erful decoding approach that leverages predic-009 tions from smaller language models (LMs) to 010 achieve both decoding acceleration and quality 011 improvement. Extensive evaluations and anal-012 yses on four diverse language tasks demon-013 strate the effectiveness of SCD, showing that 014 decoding efficiency and quality can compati-015 bly benefit from one smaller LM. 016 1 Introduction 017 Large language models (LLMs) have advanced 018 the versatility and proficiency in approaching real-019 world natural language tasks such as general in-020