ACL Findings2026
Timesteps of Mamba Align with Human Reading Times
Yuji Yamamoto, Shinnosuke Isono, Yoshinobu Kawahara, Sho Yokoi
Abstract
This study demonstrates an alignment of perword processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep ∆ t , determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with everupdated memory, because it allows us to look at how each module (layer) weighs short-and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on6
- Random-Access Infinite Context Length for TransformersAmirkeivan Mohtashami, Martin JaggiNeurIPS 2023 · 207 citations
- Context Limitations Make Neural Language Models More Human-LikeTatsuki Kuribayashi, Yohei Oseki, Ana Brassard, Kentaro InuiEMNLP 2022 · 28 citations
- If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?Ryo Yoshida, Shinnosuke Isono, Kohei Kajikawa, Taiga Someya et al.ACL 2025 · 3 citations
- Information Integration in Large Language Models is Gated by Linguistic Structural MarkersWei Liu, Nai DingEMNLP 2025
- Lower Perplexity is Not Always Human-LikeTatsuki Kuribayashi, Yohei Oseki, Takumi Ito, Ryo Yoshida et al.ACL 2021
Related papers
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space DualityTri Dao, Albert GuICML 2024 · 1,407 citations
- Temperature-scaling surprisal estimates improve fit to human reading times - but does it do so for the "right reasons"?Tong Liu, Iza Skrjanec, Vera DembergACL 2024 · 3 citations
- Probing for Reading TimesEleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re, Tianyang Xu et al.ACL 2026
- Dual Alignment Between Language Model Layers and Human Sentence ProcessingTatsuki Kuribayashi, Alex Warstadt, Yohei Oseki, Ethan Gotlieb WilcoxACL 2026 · 1 citation
- Sequential Parallel Duality in Prefix Scannable ModelsMorris Yau, Sharut Gupta, Valerie Engelmayer, Kazuki Irie et al.ICLR 2026 · 9 citations
