Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models

Large language models (LLMs) achieve impressive performance on complex mathematical benchmarks yet sometimes fail on basic math reasoning while generating unnecessarily verbose responses.In this paper, we present LLMTHINKBENCH, a systematic benchmark and comprehensive empirical study to evaluate the efficiency of reasoning in LLMs, focusing on the fundamental tradeoff between accuracy and overthinking.First, we formalize the accuracy-verbosity tradeoff.Second, we introduce the Overthinking Score, a harmonic-mean metric combining accuracy and token-efficiency for holistic model evaluation.Third, we establish an evaluation protocol with dynamically-generated data across 14 basic math tasks.Fourth, we conduct a large-scale empirical study evaluating 53 LLMs, including reasoning and quantized variants across different reasoning budgets.Fifth, we release LLM-THINKBENCH as an open-source Python package and public leaderboard for reproducibility.Our findings reveal: 1) model performance on complex benchmarks does not translate directly to basic math reasoning; 2) reasoning models generate ∼18× more tokens while sometimes achieving lower accuracy and exhibit catastrophic collapse when tokens are constrained, dropping by up to ∼36%; 3) the accuracyverbosity relationship is non-monotonic with extended reasoning budgets yielding diminishing returns (GPT-5/o-series models show zero accuracy gain from low → medium → high reasoning effort).Our findings challenge the assumption that longer reasoning in LLMs necessarily improves mathematical reasoning.0 .5 B 0 .6 B 1 B 1 B 1 B 3 B 4 B 7 B 8 B 1 4 B 3 2 B 7 0 B 7 2 B Model Parameters (B) 0 20 40 60 80 100 Accuracy (%) (a) Model Family Scaling Comparison Llama Qwen2.5 Qwen3 0 .6 B 1 .7 B 4 B 8 B 1 4 B 3 2 B Model Parameters 0 20 40 60 80 100 Accuracy (%) (b) Token Budget Impact Full Token Budget Constrained (1024 tokens) 50.0 70.2 81.9 82.1 86.5 84.1 27.0 30.7 31.7 28.4 35.2 30.2 0 .5 B 1 .5 B 3 B 7 B 1 4 B 3 2 B 7 2 B Model Parameters 0 20 40 60 80 100 Accuracy (%) (c) Quantization Impact FP16 (Original) GPTQ 8-bit GPTQ 4-bit