Vector Quantization in the Brain: Grid-like Codes in World Models
Xiangyuan Peng, Xingsi Dong, Si Wu
摘要
We propose Grid-like Code Quantization (GCQ), a brain-inspired method for compressing observation-action sequences into discrete representations using grid-like patterns in attractor dynamics. Unlike conventional vector quantization approaches that operate on static inputs, GCQ performs spatiotemporal compression through an action-conditioned codebook, where codewords are derived from continuous attractor neural networks and dynamically selected based on actions. This enables GCQ to jointly compress space and time, serving as a unified world model. The resulting representation supports long-horizon prediction, goal-directed planning, and inverse modeling. Experiments across diverse tasks demonstrate GCQ's effectiveness in compact encoding and downstream performance. Our work offers both a computational tool for efficient sequence modeling and a theoretical perspective on the formation of grid-like codes in neural systems.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper16
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Leveraging Procedural Generation to Benchmark Reinforcement LearningKarl Cobbe, Christopher Hesse, Jacob Hilton, John SchulmanICML 2020 · 被引用 685 次
- Finite Scalar Quantization: VQ-VAE Made SimpleFabian Mentzer, David Minnen, Eirikur Agustsson, Michael TschannenICLR 2024 · 被引用 442 次
- Language Modeling Is CompressionGrégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt 等ICLR 2024 · 被引用 243 次
相关 Paper
- Prediction and generalisation over directed actions by grid cellsChangmin Yu, Timothy Behrens, Neil BurgessICLR 2021 · 被引用 14 次
- Shaping Sequence Attractor Schema in Recurrent Neural NetworksZhikun Chu, Bo Ho, Xiaolong Zou, Yuanyuan MiNeurIPS 2025
- Unfolding the Black Box of Recurrent Neural Networks for Path IntegrationTianhao Chu, Yuling Wu, Neil Burgess, Zilong Ji 等NeurIPS 2025 · 被引用 6 次
- Long-Horizon Visual Planning with Goal-Conditioned Hierarchical PredictorsKarl Pertsch, Oleh Rybkin, Frederik Ebert, Shenghao Zhou 等NeurIPS 2020 · 被引用 96 次
- Efficient Planning in a Compact Latent Action SpaceZhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li 等ICLR 2023 · 被引用 3 次
