Language as a Wave Phenomenon: Semantic Phase Locking and Interference in Neural Networks
Alper YILDIRIM, İbrahim Yücedağ
摘要
In standard Transformer architectures, semantic importance is often conflated with activation magnitude, obscuring the geometric structure of latent representations. To disentangle these factors, we introduce PRISM, a complex-valued architecture designed to isolate the computational role of phase. By enforcing a strict unit-norm constraint (|z| = 1) and replacing attention with gated harmonic convolutions, the model is encouraged to utilize subtractive interference in the frequency domain to suppress noise, rather than relying on magnitude-based gating. We utilize this constrained regime to study a hybrid architecture—fusing phase-based routing with standard attention—which achieves improved parameter efficiency and representation quality compared to baselines in our evaluated settings. Mechanistically, interventional ablations indicate that the model carries substantial task-relevant information in phase: preserving phase largely maintains performance, whereas disrupting phase causes severe degradation. Together, these results suggest that phase-based spectral interference is a usable computational mechanism for neural sequence modeling at the evaluated scale.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper2
相关 Paper
- Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMsXiaoran Liu, Yuerong Song, Zhigeng Liu, Zengfeng Huang 等ICLR 2026 · 被引用 2 次
- Multiplicative Interactions and Where to Find ThemSiddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Schwarz 等ICLR 2020 · 被引用 152 次
- Prism: Spectral-Aware Block-Sparse AttentionXinghao Wang, Pengyu Wang, Xiaoran Liu, Fangxu Liu 等ICML 2026 · 被引用 3 次
- Primphormer: Efficient Graph Transformers with Primal RepresentationsMingzhen He, Ruikai Yang, Hanling Tian, Youmei Qiu 等ICML 2025
- Long-range Sequence Modeling with Predictable Sparse AttentionYimeng Zhuang, Jing Zhang, Mei TuACL 2022 · 被引用 11 次
