One Bad Token Spoils the Barrel: Assessment, Detection, and Remediation of Glitch Tokens in Large Language Models

Large Language Models (LLMs) have shown remarkable capabilities across numerous applications. However, the recent emergence of "glitch tokens," referring to tokens that cause unpredictable and erroneous model behaviors, poses significant reliability and security challenges. Despite prior investigations of glitch tokens, critical gaps remain, including insufficient safety assessments, limited detection coverage, and the absence of effective remediation strategies. To address these challenges, we first demonstrate that glitch tokens can readily bypass safety mechanisms and elicit unsafe outputs across LLMs through straightforward exploitation methods. More critically, we reveal that these tokens exhibit cross-model transferability, inducing safety risks across model families, tokenizers, commercial and moderation systems, underscoring their pervasive security implications. We then propose GlitchQuiz, a red-teaming framework inspired by human language acquisition, to comprehensively detect glitch tokens, surpassing existing detection methods. Finally, we develop GlitchEdit, a training-free embedding-layer editing approach that effectively remediates glitch tokens, reducing unsafe behaviors with average unsafe rates decreasing from 96.36% to 2.87% across evaluated LLMs and maintaining overall performance. Our findings have been responsibly disclosed to nine affected leading LLM providers, including OpenAI, Anthropic, and others, to help foster safer AI ecosystems.

Warning: This paper contains examples of unsafe content generated by large language models, including violence, discrimination, and other content that may be disturbing or offensive to some readers.