Lune

ACL Findings2023

Mapping Brains with Language Models: A Survey

Antonia Karamolegkou, Mostafa Abdou, Anders Søgaard

Abstract

Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models. In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics. How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions? Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative. We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
  • Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Joint processing of linguistic properties in brains and language models
  • Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models cites Improving Semantic Understanding in Speech Language Models via Brain-tuning
  • When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models
  • When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Joint processing of linguistic properties in brains and language models
  • When Language Models Lose Their Mind: The Consequences of Brain Misalignment cites Improving Semantic Understanding in Speech Language Models via Brain-tuning
  • Toward a realistic model of speech processing in the brain with self-supervised learning cites Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech
  • Toward a realistic model of speech processing in the brain with self-supervised learning cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
  • Toward a realistic model of speech processing in the brain with self-supervised learning cites Disentangling syntax and semantics in the brain with deep networks
  • Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech
  • Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Low-dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
  • Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Joint processing of linguistic properties in brains and language models
  • Improving Semantic Understanding in Speech Language Models via Brain-tuning cites Toward a realistic model of speech processing in the brain with self-supervised learning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language ModelsAnuja Negi, Subba Reddy Oota, Anwar Nunez-Elizalde, Manish Gupta et al.NeurIPS 2025 · 9 citations
  2. When Language Models Lose Their Mind: The Consequences of Brain MisalignmentGabriele Merlin, Mariya TonevaICLR 2026 · 3 citations
  3. Do Language Models Have Semantics? On the Five Standard PositionsAnders SøgaardACL 2025 · 2 citations
  4. Improving Semantic Understanding in Speech Language Models via Brain-tuningOmer Moussa, Dietrich Klakow, Mariya TonevaICLR 2025

Builds on10

  1. Toward a realistic model of speech processing in the brain with self-supervised learningJuliette Millet, Charlotte Caucheteux, Pierre Orhan, Yves Boubenec et al.NeurIPS 2022 · 164 citations
  2. Disentangling syntax and semantics in the brain with deep networksCharlotte Caucheteux, Alexandre Gramfort, Jean-Remi KingICML 2021 · 96 citations
  3. Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to SpeechAditya R. Vaidya, Shailee Jain, Alexander HuthICML 2022 · 81 citations
  4. Joint processing of linguistic properties in brains and language modelsSubba Reddy Oota, Manish Gupta, Mariya TonevaNeurIPS 2023 · 64 citations
  5. Low-dimensional Structure in the Space of Language Representations is Reflected in Brain ResponsesRichard J. Antonello, Javier S. Turek, Vy Ai Vo, Alexander HuthNeurIPS 2021 · 60 citations

Related papers

  1. From Language to Cognition: How LLMs Outgrow the Human Language NetworkBadr AlKhamissi, Greta Tuckute, Yingtian Tang, Taha Osama A Binhuraib et al.EMNLP 2025 · 1 citation
  2. Abstraction Induces the Brain Alignment of Language and Speech ModelsEmily Cheng, Aditya Vaidya, Richard AntonelloICML 2026
  3. Do Large Language Models Think like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRIYu Lei, Xingyang Ge, Yi Zhang, Yiming Yang et al.AAAI 2026 · 2 citations
  4. Neural Language Models are not Born Equal to Fit Brain Data, but Training HelpsAlexandre Pasquiou, Yair Lakretz, John T. Hale, Bertrand Thirion et al.ICML 2022 · 44 citations
  5. Numeric Magnitude Comparison Effects in Large Language ModelsRaj Sanjay Shah, Vijay Marupudi, Reba Koenen, Khushi Bhardwaj et al.ACL Findings 2023 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines