Lune

ACL Findings2023

Numeric Magnitude Comparison Effects in Large Language Models

Raj Sanjay Shah, Vijay Marupudi, Reba Koenen, Khushi Bhardwaj, Sashank Varma

Abstract

Large Language Models (LLMs) do not differentially represent numbers, which are pervasive in text. In contrast, neuroscience research has identified distinct neural representations for numbers and words. In this work, we investigate how well popular LLMs capture the magnitudes of numbers (e.g., that 4 < 5) from a behavioral lens. Prior research on the representational capabilities of LLMs evaluates whether they show human-level performance, for instance, high overall accuracy on standard benchmarks. Here, we ask a different question, one inspired by cognitive science: How closely do the number representations of LLMs correspond to those of human language users, who typically demonstrate the distance, size, and ratio effects? We depend on a linking hypothesis to map the similarities among the model embeddings of number words and digits to human response times. The results reveal surprisingly human-like representations across language models of different architectures, despite the absence of the neural circuitry that directly supports these representations in the human brain. This research shows the utility of understanding LLMs using behavioral benchmarks and points the way to future work on the number of representations of LLMs and their cognitive plausibility.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

5 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers5

  1. Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?Andreas Opedal, Alessandro Stolfo, Haruki Shirakami, Ying Jiao et al.ICML 2024 · 27 citations
  2. Understanding Counting Mechanisms in Large Language and Vision-Language ModelsHosein Hasani, Amirmohammad Izadi, Fatemeh Askari, Mobin Bagherian et al.CVPR 2026 · 5 citations
  3. The Illusion of Generalization in Tabular Language ModelsAditya Gorla, Ratish Surendran PuduppullyICML 2026 · 2 citations
  4. Computer Vision Modeling of the Development of Geometric and Numerical Concepts in HumansZekun Wang, Sashank VarmaAAAI 2026
  5. Comparing the learning dynamics of in-context learning and fine-tuning in language modelsBasile Confavreux, Aaditya K Singh, Jin Hwa Lee, Amaury Sabran et al.ICLR 2026

Builds on1

  1. Injecting Numerical Reasoning Skills into Language ModelsMor Geva, Ankit Gupta, Jonathan BerantACL 2020 · 12 citations

Related papers

  1. Language Models Learn Universal Representations of Numbers and Here's Why You Should CareMichal Stefánik, Timothee Mickus, Marek Kadlcík, Bertram Højer et al.ACL 2026 · 1 citation
  2. Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language ModelsHaoyang Li, Xuejia Chen, Zhanchao Xu, Darian Li et al.ACL Findings 2025 · 12 citations
  3. Conceptual structure coheres in human cognition but not in large language modelsSiddharth Suresh, Kushin Mukherjee, Xizheng Yu, Wei-Chun Huang et al.EMNLP 2023 · 7 citations
  4. Do Large Language Models Think like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRIYu Lei, Xingyang Ge, Yi Zhang, Yiming Yang et al.AAAI 2026 · 2 citations
  5. VisNumBench: Evaluating Number Sense of Multimodal Large Language ModelsTengjin Weng, Jingyi Wang, Wenhao Jiang, Zhong MingICCV 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines