Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
Tatiana Petrova, Evgeny Polyachenko, Radu State
Abstract
We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity , comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an -sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads . For LSR, the finite support introduces a threshold below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 8815a299-4eed-48cc-b86e-c1ecd49227d5Builds on2
Related papers
- Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical CodesJerry Yao-Chieh Hu, Dennis Wu, Han LiuNeurIPS 2024 · 26 citations
- On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity AnalysisJerry Yao-Chieh Hu, Thomas Lin, Zhao Song, Han LiuICML 2024 · 47 citations
- Kernel Memory Networks: A Unifying Framework for Memory ModelingGeorgios Iatropoulos, Johanni Brea, Wulfram GerstnerNeurIPS 2022 · 15 citations
- Dynamical properties of dense associative memoryKazushi Mimura, Jun'ichi Takeuchi, Yuto Sumikawa, Yoshiyuki Kabashima et al.ICLR 2026 · 6 citations
- Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory ModelsBeren Millidge, Tommaso Salvatori, Yuhang Song, Thomas Lukasiewicz et al.ICML 2022 · 72 citations
