Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning
Valentin NOËL
Abstract
Verifying whether a language model is genuinely reasoning or pattern-matching remains an open problem: learned verifiers are expensive, and output-based heuristics are brittle. We show that valid mathematical reasoning induces a measurable, training-free spectral signature in transformer attention. By treating each attention matrix as a weighted token graph, we extract four diagnostics: Fiedler value, High-Frequency Energy Ratio (HFER), spectral entropy, and smoothness, that require no learned parameters. Experiments across seven models from four architectural families yield effect sizes up to Cohen's (), enabling -% single-threshold classification accuracy. Two findings sharpen the interpretation. First, Platonic validity: the spectral signal tracks logical coherence rather than compiler acceptance, proofs rejected for timeouts or missing imports are correctly classified as valid, a distinction confirmed by a manual audit (, ). Second, architectural determinism: Sliding Window Attention shifts the discriminative feature from HFER to smoothness (, ), showing that attention design governs which spectral channel encodes reasoning quality. Causal ablation confirms the signature traces induction-head circuits. The method generalises to informal chain-of-thought (, ), and in proof search, HFER reranking improves Best-of-16 Pass@1 by -%, matching % of the AUC of fully supervised probes with zero labels. Spectral graph analysis is a principled, architecture-aware primitive for reasoning verification.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 629319a9-7e24-487f-8b87-8082df6455ebBuilds on21
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer et al.NeurIPS 2022 · 2,039 citations
- Rethinking Graph Transformers with Spectral AttentionDevin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau et al.NeurIPS 2021 · 854 citations
- Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li et al.NeurIPS 2023 · 728 citations
Related papers
- Verifying Chain-of-Thought Reasoning via Its Computational GraphZheng Zhao, Yeskendir Koishekenov, Xianjun Yang, Naila Murray et al.ICLR 2026 · 25 citations
- HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMsAzim Ospanov, Zijin Feng, Jiacheng Sun, Haoli Bai et al.ICML 2026 · 5 citations
- RFS-Guard: Detecting Reasoning Hallucinations via Cross-Phase Routing Focus in Large Reasoning ModelsZihang Liu, Zhouhua Fang, Hui Liu, Zhiwei Liu et al.ACL 2026
- Reasoning Structure of Large Language ModelsFrédéric Berdoz, Luca Lanzendörfer, Fabian Farestam, Roger WattenhoferICML 2026
- Who Reasons in the Large Language Models?Jie Shao, Jianxin WuNeurIPS 2025 · 7 citations
