The Hippocampal Place Field Gradient: A Bio-inspired Framework Building Multiscale Representation for Better Sample Efficiency
Shujun Zhou, Junrong Qi, Guozhang Chen
Abstract
The hippocampus encodes space through a striking gradient of place field sizes along its dorsal-ventral axis, yet the principles generating this continuous gradient from discrete grid cell inputs remain unclear. We propose a unified theoretical framework establishing how multiscale hippocampal place fields arise from the frequency-dependent decay of grid cell projections. Functionally, this organization establishes an inductive bias in the population code, managing a fundamental trade-off between spatial precision and sample efficiency. Translating this insight to artificial neural networks, we incorporate a hippocampus-inspired positional embedding (HIPE) into the Transformer architecture to induce multi-scale representation. Experimental results confirm that this mechanism effectively improves data efficiency. Our work establishes a link between neural connectivity, activity patterns, and learning, suggesting a principled approach to utilizing multi-scale representations for sample-efficiency learning. Our codes are available at https://github.com/AIogry/relative_PE.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on8
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra et al.NeurIPS 2022 · 5,493 citations
- Train Short, Test Long: Attention with Linear Biases Enables Input Length ExtrapolationOfir Press, Noah A. Smith, Mike LewisICLR 2022 · 1,168 citations
- Relating transformers to models and neural representations of the hippocampal formationJames C. R. Whittington, Joseph Warren, Tim E. J. BehrensICLR 2022 · 110 citations
- OLMo: Accelerating the Science of Language ModelsDirk Groeneveld, Iz Beltagy, Evan Pete Walsh, Akshita Bhagia et al.ACL 2024 · 52 citations
Related papers
- Hippoformer: Integrating Hippocampus-inspired Spatial Memory with TransformersTiantian Li, Xingxing Cao, Yifei Wang, Xiaojiao Yang et al.ICLR 2026
- Place Cells as Multi-Scale Position Embeddings: Random Walk Transition Kernels for Path PlanningMinglu Zhao, Dehong Xu, Deqian Kong, Wenhao Zhang et al.NeurIPS 2025 · 1 citation
- Positional Encoding for Spiking TransformersZijian Zhou, Yu Liang, Honglin Cao, Ammar Belatreche et al.ICML 2026 · 7 citations
- ResT: An Efficient Transformer for Visual RecognitionQinglong Zhang, Yu-Bin YangNeurIPS 2021 · 313 citations
- A Multi-Region Brain Model to Elucidate the Role of Hippocampus in Spatially Embedded Decision-MakingYi Xie, Jaedong Hwang, Carlos D. Brody, David W. Tank et al.ICML 2025
