Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones

Despite remarkable advances in coding capabilities, language models (LMs) still struggle with simple syntactic tasks such as generating balanced parentheses. In this study, we investigate the underlying mechanisms behind the persistence of these errors across LMs of varying sizes (124M-7B) to both understand and mitigate the errors. Our study reveals that LMs rely on a number of components (attention heads and FF neurons) that independently make their own predictions. While some components reliably predict correct answers across a generalized range of inputs (i.e., implementing "sound mechanisms"), others are less reliable and introduce noise by promoting incorrect tokens (i.e., implementing "faulty mechanisms"). Errors occur when the faulty mechanisms overshadow the sound ones and dominantly affect the predictions. Motivated by this insight, we introduce RAS-TEER, a steering method to systematically identify and increase the contribution of reliable components for improving model performance. RASTEER substantially improves performance on balanced parentheses tasks, boosting accuracy of some models from 0% to around 100%, without impairing the models' general coding ability. We further demonstrate its broader applicability in arithmetic reasoning tasks, achieving performance gains of up to around 20%.

where

V alue are the query, key, and value matrices computed from the input r ℓ using learned projection matrices W ℓ,h Query , W ℓ,h Key , W ℓ,h V alue ∈ R d×dhead and W ℓ,h O ∈ R dhead×d are learned projection matrices specific to head (ℓ, h), and d head is the dimensionality per head.

Feed-forward (FF) Sub-layer The FF sub-layer at each layer ℓ consists of a two-layer feed-forward network: