ACL Findings2026
Mechanistic Insights into Deferred Semantic Drift in LLMs
Jingjie Zeng, Huayang Li, Liang Yang, Shaowu Zhang, Yuanyuan Sun, Hongfei Lin
Abstract
Large Language Models (LLMs) face a fundamental challenge with delayed disambiguation: How is the meaning of an ambiguous word updated when clarifying context arrives only after it has been processed? While LLMs possess the latent capacity to resolve such ambiguities-as revealed when a full, non-causal context is provided-their unidirectional architecture prevents immediate updates. We investigate the underlying computational mechanism and show this semantic re-evaluation is deferred to subsequent tokens in a process we term "Deferred Semantic Drift (DSD)". Through targeted analysis of attentional pathways, we find that later tokens actively retrieve context-dependent "informational packets" from the ambiguous word's value vector to steer the final interpretation. We demonstrate this mechanism in metaphor comprehension and provide causal validation by steering model outputs towards literal or metaphorical meanings via targeted activation interventions. This research uncovers a key computational strategy for meaning construction, offering crucial insights for understanding and guiding the behavior of LLMs 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on11
- Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
- Investigating Gender Bias in Language Models Using Causal Mediation AnalysisJesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian et al.NeurIPS 2020 · 851 citations
- Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language ModelsAsma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon et al.ICML 2024 · 197 citations
- Interpreting CLIP's Image Representation via Text-Based DecompositionYossi Gandelsman, Alexei A. Efros, Jacob SteinhardtICLR 2024 · 179 citations
- Discovering Latent Concepts Learned in BERTFahim Dalvi, Abdul Rafae Khan, Firoj Alam, Nadir Durrani et al.ICLR 2022 · 74 citations
Related papers
- DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain TranslationZhibo Man, Yuanmeng Chen, Yujie Zhang, Jinan XuEMNLP 2025
- Label Words are Anchors: An Information Flow Perspective for Understanding In-Context LearningLean Wang, Lei Li, Damai Dai, Deli Chen et al.EMNLP 2023 · 26 citations
- Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A SurveyMd. Mehrab Tanjim, Yeonjun In, Xiang Chen, Victor S. Bursztyn et al.EMNLP 2025 · 2 citations
- Dual Alignment Between Language Model Layers and Human Sentence ProcessingTatsuki Kuribayashi, Alex Warstadt, Yohei Oseki, Ethan Gotlieb WilcoxACL 2026 · 1 citation
- Attention Sinks: A 'Catch, Tag, Release' Mechanism for EmbeddingsStephen Zhang, Mustafa Khan, Vardan PapyanNeurIPS 2025 · 18 citations
