Sparse Activation Editing for Reliable Instruction Following in Narratives

Complex narrative contexts often challenge language models' ability to follow instructions, and existing benchmarks fail to capture these difficulties. To address this, we propose Concise-SAE, a training-free framework that improves instruction following by identifying and editing instruction-relevant neurons using only natural language instructions, without requiring labelled data. To thoroughly evaluate our method, we introduce FREEIN-STRUCT, a diverse and realistic benchmark of 1,212 examples that highlights the challenges of instruction following in narrative-rich settings. While initially motivated by complex narratives, Concise-SAE demonstrates stateof-the-art instruction adherence across varied tasks without compromising generation quality. The data and code are available at https: //github.com/Chacioc/Concise-SAE .