The Hippocampal Place Field Gradient: A Bio-inspired Framework Building Multiscale Representation for Better Sample Efficiency
Shujun Zhou, Junrong Qi, Guozhang Chen
摘要
The hippocampus encodes space through a striking gradient of place field sizes along its dorsal-ventral axis, yet the principles generating this continuous gradient from discrete grid cell inputs remain unclear. We propose a unified theoretical framework establishing how multiscale hippocampal place fields arise from the frequency-dependent decay of grid cell projections. Functionally, this organization establishes an inductive bias in the population code, managing a fundamental trade-off between spatial precision and sample efficiency. Translating this insight to artificial neural networks, we incorporate a hippocampus-inspired positional embedding (HIPE) into the Transformer architecture to induce multi-scale representation. Experimental results confirm that this mechanism effectively improves data efficiency. Our work establishes a link between neural connectivity, activity patterns, and learning, suggesting a principled approach to utilizing multi-scale representations for sample-efficiency learning. Our codes are available at https://github.com/AIogry/relative_PE.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper8
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel 等ICLR 2020 · 被引用 7,418 次
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra 等NeurIPS 2022 · 被引用 5,493 次
- Train Short, Test Long: Attention with Linear Biases Enables Input Length ExtrapolationOfir Press, Noah A. Smith, Mike LewisICLR 2022 · 被引用 1,168 次
- Relating transformers to models and neural representations of the hippocampal formationJames C. R. Whittington, Joseph Warren, Tim E. J. BehrensICLR 2022 · 被引用 110 次
- OLMo: Accelerating the Science of Language ModelsDirk Groeneveld, Iz Beltagy, Evan Pete Walsh, Akshita Bhagia 等ACL 2024 · 被引用 52 次
相关 Paper
- Hippoformer: Integrating Hippocampus-inspired Spatial Memory with TransformersTiantian Li, Xingxing Cao, Yifei Wang, Xiaojiao Yang 等ICLR 2026
- Place Cells as Multi-Scale Position Embeddings: Random Walk Transition Kernels for Path PlanningMinglu Zhao, Dehong Xu, Deqian Kong, Wenhao Zhang 等NeurIPS 2025 · 被引用 1 次
- Positional Encoding for Spiking TransformersZijian Zhou, Yu Liang, Honglin Cao, Ammar Belatreche 等ICML 2026 · 被引用 7 次
- ResT: An Efficient Transformer for Visual RecognitionQinglong Zhang, Yu-Bin YangNeurIPS 2021 · 被引用 313 次
- A Multi-Region Brain Model to Elucidate the Role of Hippocampus in Spatially Embedded Decision-MakingYi Xie, Jaedong Hwang, Carlos D. Brody, David W. Tank 等ICML 2025
