Towards Practical LLM Unlearning: Efficient, Modular, and Retain-Free

Large Language Models (LLMs), trained on vast web corpora and now widely integrated into web services like search engines and chatbots, have raised growing concerns about their privacy and security. This has spurred the development of Machine Unlearning techniques, which aim to effectively remove the influence of specific data (such as private user information, copyrighted web content, or harmful knowledge) from trained models while preserving general performance. However, existing LLM unlearning approaches face significant challenges, including extensive parameter modifications, a heavy reliance on retain sets to preserve utility, and limited scalability for handling sequential unlearning requests, impeding their real-world deployment in dynamic web environments. In this work, we propose Semantic Redirection for Unlearning (SRU), a lightweight framework that fine-tunes only the output word embedding layer. This targeted adjustment reshapes the model's semantic-to-lexical mapping to block undesired concepts without disturbing deeper representations, thereby preserving the model's general performance and eliminating dependency on retain sets. Furthermore, SRU's modular design enables independent, sequential unlearning tasks—a vital feature for live web services handling continuous data removal requests. Experiments on the MUSE benchmark demonstrate that SRU achieves state-of-the-art efficiency, reducing computational cost by approximately 98%, while maintaining competitive unlearning performance, making it a practical and efficient solution for building compliant and trustworthy LLM-powered web applications.