ACL Findings2023
Mapping Brains with Language Models: A Survey
Antonia Karamolegkou, Mostafa Abdou, Anders Søgaard
Abstract
Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models. In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics. How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions? Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative. We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
- Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Joint processing of linguistic properties in brains and language models
- Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Improving Semantic Understanding in Speech Language Models via Brain-tuning
- When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models
- When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Joint processing of linguistic properties in brains and language models
- When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Improving Semantic Understanding in Speech Language Models via Brain-tuning
- Toward a realistic model of speech processing in the brain with self-supervised learning cites Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech
- Toward a realistic model of speech processing in the brain with self-supervised learning cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
- Toward a realistic model of speech processing in the brain with self-supervised learning cites Disentangling syntax and semantics in the brain with deep networks
- Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech
- Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
- Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Joint processing of linguistic properties in brains and language models
- Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Toward a realistic model of speech processing in the brain with self-supervised learning
Cited by top-tier papers4
- Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language ModelsAnuja Negi, Subba Reddy Oota, Anwar Nunez-Elizalde, Manish Gupta et al.NeurIPS 2025 · 9 citations
- When Language Models Lose Their Mind: The Consequences of Brain MisalignmentGabriele Merlin, Mariya TonevaICLR 2026 · 3 citations
- Do Language Models Have Semantics? On the Five Standard PositionsAnders SøgaardACL 2025 · 2 citations
- Improving Semantic Understanding in Speech Language Models via Brain-tuningOmer Moussa, Dietrich Klakow, Mariya TonevaICLR 2025
Builds on10
- Toward a realistic model of speech processing in the brain with self-supervised learningJuliette Millet, Charlotte Caucheteux, Pierre Orhan, Yves Boubenec et al.NeurIPS 2022 · 164 citations
- Disentangling syntax and semantics in the brain with deep networksCharlotte Caucheteux, Alexandre Gramfort, Jean-Remi KingICML 2021 · 96 citations
- Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to SpeechAditya R. Vaidya, Shailee Jain, Alexander HuthICML 2022 · 81 citations
- Joint processing of linguistic properties in brains and language modelsSubba Reddy Oota, Manish Gupta, Mariya TonevaNeurIPS 2023 · 64 citations
- Low-dimensional Structure in the Space of Language Representations is Reflected in Brain ResponsesRichard J. Antonello, Javier S. Turek, Vy Ai Vo, Alexander HuthNeurIPS 2021 · 60 citations
Related papers
- From Language to Cognition: How LLMs Outgrow the Human Language NetworkBadr AlKhamissi, Greta Tuckute, Yingtian Tang, Taha Osama A Binhuraib et al.EMNLP 2025 · 1 citation
- Abstraction Induces the Brain Alignment of Language and Speech ModelsEmily Cheng, Aditya Vaidya, Richard AntonelloICML 2026
- Do Large Language Models Think like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRIYu Lei, Xingyang Ge, Yi Zhang, Yiming Yang et al.AAAI 2026 · 2 citations
- Neural Language Models are not Born Equal to Fit Brain Data, but Training HelpsAlexandre Pasquiou, Yair Lakretz, John T. Hale, Bertrand Thirion et al.ICML 2022 · 44 citations
- Numeric Magnitude Comparison Effects in Large Language ModelsRaj Sanjay Shah, Vijay Marupudi, Reba Koenen, Khushi Bhardwaj et al.ACL Findings 2023 · 4 citations
