Lune

ACL Findings2024

LLMs cannot find reasoning errors, but can correct them given the error location

Gladys Tyen, Hassan Mansoor, Victor Carbune, Peter Chen, Tony Mak

Abstract

While self-correction has shown promise in improving LLM outputs in terms of style and quality (e.g. Chen et al., 2023b;Madaan et al., 2023), recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in worse performances overall (Huang et al., 2023). In this paper, we show that poor self-correction performance stems from LLMs' inability to find logical mistakes, rather than their ability to correct a known mistake. Firstly, we benchmark several state-of-the-art LLMs on their mistake-finding ability and demonstrate that they generally struggle with the task, even in highly objective, unambiguous cases. Secondly, we test the correction abilities of LLMs -separately from mistake finding -using a backtracking setup that feeds ground truth mistake location information to the model. We show that this boosts downstream task performance across our 5 reasoning tasks, indicating that LLMs' correction abilities are robust. Finally, we show that it is possible to obtain mistake location information without ground truth labels or in-domain training data. We train a small classifier with out-of-domain data, which exhibits stronger mistake-finding performance than prompting a large model. We release our dataset of LLM-generated logical mistakes, BIG-Bench Mistake, to enable further research into locating LLM reasoning mistakes.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

51 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Sherlock: Self-Correcting Reasoning in Vision-Language Models cites Self-Refine: Iterative Refinement with Self-Feedback
  • Sherlock: Self-Correcting Reasoning in Vision-Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales? cites Self-Refine: Iterative Refinement with Self-Feedback
  • Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales? cites Reflexion: language agents with verbal reinforcement learning
  • Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales? cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation cites Self-Refine: Iterative Refinement with Self-Feedback
  • Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation cites Reflexion: language agents with verbal reinforcement learning
  • Self-Refine: Iterative Refinement with Self-Feedback cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Refine: Iterative Refinement with Self-Feedback cites Training language models to follow instructions with human feedback
  • S^3cMath: Spontaneous Step-Level Self-Correction Makes Large Language Models Better Mathematical Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Reflexion: language agents with verbal reinforcement learning cites Self-Refine: Iterative Refinement with Self-Feedback
  • Reflexion: language agents with verbal reinforcement learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Verifying Chain-of-Thought Reasoning via Its Computational Graph cites Let's Verify Step by Step
  • Verifying Chain-of-Thought Reasoning via Its Computational Graph cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Let's Verify Step by Step cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Let's Verify Step by Step cites Training language models to follow instructions with human feedback
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers51

  1. Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo et al.NeurIPS 2024 · 74 citations
  2. Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI AutomationYuyang Wanyan, Xi Zhang, Haiyang Xu, Haowei Liu et al.NeurIPS 2025 · 26 citations
  3. Verifying Chain-of-Thought Reasoning via Its Computational GraphZheng Zhao, Yeskendir Koishekenov, Xianjun Yang, Naila Murray et al.ICLR 2026 · 25 citations
  4. S^3cMath: Spontaneous Step-Level Self-Correction Makes Large Language Models Better Mathematical ReasonersYuchen Yan, Jin Jiang, Yang Liu, Yixin Cao et al.AAAI 2025 · 19 citations
  5. Sherlock: Self-Correcting Reasoning in Vision-Language ModelsYi Ding, Ruqi ZhangNeurIPS 2025 · 14 citations

Builds on18

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  4. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  5. Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations

Related papers

  1. Large Language Models Cannot Self-Correct Reasoning YetJie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng et al.ICLR 2024 · 858 citations
  2. CriticBench: Benchmarking LLMs for Critique-Correct ReasoningZicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo et al.ACL Findings 2024 · 16 citations
  3. Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for ReasoningYongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang et al.ACL 2024
  4. LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language ModelsMihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura et al.ACL 2024
  5. Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical ReasoningJoykirat Singh, Akshay Uttama Nambi, Vibhav VineetACL 2025 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines