Reasoning with Latent Thoughts: On the Power of Looped Transformers
Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi
Abstract
Large language models have shown remarkable reasoning abilities and scaling laws suggest that large parameter count, especially along the depth axis, is the primary driver. In this work, we make a stronger claim -many reasoning problems require a large depth but not necessarily many parameters. This unlocks a novel application of looped models for reasoning. Firstly, we show that for many synthetic reasoning problems like addition, p-hop induction, and math problems, a k-layer transformer looped L times nearly matches the performance of a kL-layer non-looped model, and is significantly better than a k-layer model. This is further corroborated by theoretical results showing that many such reasoning problems can be solved via iterative algorithms, and thus, can be solved effectively using looped models with nearly optimal depth. Perhaps surprisingly, these benefits also translate to practical settings of language modeling -on many downstream reasoning tasks, a language model with k-layers looped L times can be competitive to, if not better than, a kL-layer language model. In fact, our empirical analysis reveals an intriguing phenomenon: looped and non-looped models exhibit scaling behavior that depends on their effective depth, akin to the inference-time scaling of chain-of-thought (CoT) reasoning. We further elucidate the connection to CoT reasoning by proving that looped models implicitly generate latent thoughts and can simulate T steps of CoT with T loops. Inspired by these findings, we also present an interesting dichotomy between reasoning and memorization, and design a looping-based regularization that is effective on both fronts.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
50 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- The Expressive Power of Transformers with Chain of Thought cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- The Expressive Power of Transformers with Chain of Thought cites Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
- Thinkless: LLM Learns When to Think cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain of Thought Empowers Transformers to Solve Inherently Serial Problems cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain of Thought Empowers Transformers to Solve Inherently Serial Problems cites Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
- Dynamic Early Exit in Reasoning Models cites Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains
- Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains cites Dynamic Early Exit in Reasoning Models
- Dynamic Early Exit in Reasoning Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation cites Dynamic Early Exit in Reasoning Models
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation cites Language Models are Few-Shot Learners
- Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective cites Language Models are Few-Shot Learners
- Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers50
- Dynamic Early Exit in Reasoning ModelsChenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu et al.ICLR 2026 · 250 citations
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level ComputationSangmin Bae, Yujin Kim, Reza Bayat, Sungnyun Kim et al.NeurIPS 2025 · 143 citations
- Thinkless: LLM Learns When to ThinkGongfan Fang, Xinyin Ma, Xinchao WangNeurIPS 2025 · 128 citations
- Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning ChainsWenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo et al.NeurIPS 2025 · 103 citations
- BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for ReasoningXuechen Zhang, Zijian Huang, Yingcong Li, Chenshun Ni et al.NeurIPS 2025 · 66 citations
Builds on26
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Towards Revealing the Mystery behind Chain of Thought: A Theoretical PerspectiveGuhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye et al.NeurIPS 2023 · 470 citations
- Chain of Thought Empowers Transformers to Solve Inherently Serial ProblemsZhiyuan Liu, Hong Liu, Denny Zhou, Tengyu MaICLR 2024 · 259 citations
- The Expressive Power of Transformers with Chain of ThoughtWilliam Merrill, Ashish SabharwalICLR 2024 · 243 citations
Related papers
- Analyzing the Power of Chain of Thought through Memorization CapabilitiesLijia Yu, Xiao-Shan Gao, Lijun ZhangNeurIPS 2025 · 2 citations
- Language models can learn implicit multi-hop reasoning, but only if they have lots of training dataYuekun Yao, Yupei Du, Dawei Zhu, Michael Hahn et al.EMNLP 2025
- LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut ModulationAhmadreza Jeddi, Marco Ciccone, Babak TaatiICLR 2026 · 54 citations
- A Formal Comparison Between Chain of Thought and Latent ThoughtKevin Xu, Issei SatoICML 2026 · 12 citations
- From Growing to Looping: A Unified View of Iterative Computation in LLMsFerdinand Kapl, Emmanouil Angelis, Kaitlin Maile, Johannes von Oswald et al.ICML 2026 · 2 citations
