EMNLP Findings2024
Can LLMs Learn From Mistakes? An Empirical Study on Reasoning Tasks
Shengnan An, Zexiong Ma, Siqi Cai, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, Weizhu Chen
Abstract
Towards enhancing the chain-of-thought (CoT) reasoning of large language models (LLMs), much existing work has revealed the effectiveness of straightforward learning on annotated/generated CoT paths.However, there is less evidence yet that reasoning capabilities can be enhanced through a reverse learning process, i.e., learning from potential mistakes in reasoning.To investigate whether LLMs can learn from mistakes, we construct mistakecorrection datasets, using GPT-4 to identify and correct the mistakes in inaccurate CoTs.With these mistake-correction datasets, we fine-tune open-source LLMs and arrive at the following conclusions.(1) LLMs can indeed learn from mistakes to enhance their CoT reasoning performances.(2) Compared to CoT data, the mistake-correction data provides additional knowledge on the explanations and reasons for the potential mistakes in CoTs, which consistently contributes to the effectiveness of learning from mistakes.(3) Evolution techniques, especially the correction-centric evolution we introduced, can further enhance the effectiveness of learning from mistakes. LLMQuestion: Tina makes 18.00 an hour for 8 hours, which is 8 * 144.00.Step 2: She makes 27.00 = 144.00 + 198.00.Step 4: The answer is 198. GPT-4 As CorrectorCorrection: Incorrect Step: Step 3. Explanation: Step 3 only calculates the earnings for one day, but not for the entire five days.Correct Solution: ... Step 3: For one day, she makes 54.00 = 198.00 * 5 = $990.00.Step 5: The answer is 990. Question-Answer Pairs
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
- QLoRA: Efficient Finetuning of Quantized LLMs cites LoRA: Low-Rank Adaptation of Large Language Models
- QLoRA: Efficient Finetuning of Quantized LLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
- CLER: Improving Multimodal Financial Reasoning by Cross-MLLM Error Reflection cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers1
Builds on36
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
Related papers
- Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for ReasoningYongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang et al.ACL 2024
- Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?Renbiao Liu, Anqi Li, Chaoding Yang, Hui Sun et al.ICML 2025
- Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained RewardsHyeonbin Hwang, Doyoung Kim, Seungone Kim, Seonghyeon Ye et al.EMNLP Findings 2024 · 6 citations
- Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and CorrectionXiaoyuan Li, Wenjie Wang, Moxin Li, Junrong Guo et al.ACL Findings 2024 · 15 citations
- Large Language Models are Better Reasoners with Self-VerificationYixuan Weng, Minjun Zhu, Fei Xia, Bin Li et al.EMNLP Findings 2023 · 110 citations
