Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
Tatiana Petrova, Evgeny Polyachenko, Radu State
摘要
We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity , comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an -sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads . For LSR, the finite support introduces a threshold below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper2
相关 Paper
- Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical CodesJerry Yao-Chieh Hu, Dennis Wu, Han LiuNeurIPS 2024 · 被引用 26 次
- On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity AnalysisJerry Yao-Chieh Hu, Thomas Lin, Zhao Song, Han LiuICML 2024 · 被引用 47 次
- Kernel Memory Networks: A Unifying Framework for Memory ModelingGeorgios Iatropoulos, Johanni Brea, Wulfram GerstnerNeurIPS 2022 · 被引用 15 次
- Dynamical properties of dense associative memoryKazushi Mimura, Jun'ichi Takeuchi, Yuto Sumikawa, Yoshiyuki Kabashima 等ICLR 2026 · 被引用 6 次
- Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory ModelsBeren Millidge, Tommaso Salvatori, Yuhang Song, Thomas Lukasiewicz 等ICML 2022 · 被引用 72 次
