Timesteps of Mamba Align with Human Reading Times

This study demonstrates an alignment of perword processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep ∆ t , determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with everupdated memory, because it allows us to look at how each module (layer) weighs short-and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online. 1 * Equal contribution 1 Code available at https://osf.io/vnw5e/overview? view_only=93ad704fc6ea44438f3d3538b4b682eb