ACL Findings2025
Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic
Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
Abstract
Self-critic has become a crucial mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts for intuitive instance-level feedback, which resembles System-1 processes and limits the reasoning capabilities. Moreover, there is a lack of in-depth investigations into the relationship between LLM's ability to criticize and its task-solving performance. To address these issues, we propose Critic-CoT, a novel framework that pushes LLMs toward System-2-like critic capability. Through a step-wise CoT reasoning paradigm and the automatic construction of weak-supervision data without human annotation, Critic-CoT enables LLMs to engage in slow, analytic self-critique and refinement, thereby improving their reasoning abilities. Experiments on GSM8K and MATH and out-of-domain evaluation demonstrate that our enhanced model significantly boosts tasksolving performance by filtering out invalid solutions or iterative refinement. Furthermore, we investigate the intrinsic correlation between critique and task-solving abilities within LLMs, discovering that these abilities can mutually reinforce each other rather than conflict. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Self-Refine: Iterative Refinement with Self-Feedback cites Training language models to follow instructions with human feedback
- Reflexion: language agents with verbal reinforcement learning cites Self-Refine: Iterative Refinement with Self-Feedback
- Let's Verify Step by Step cites Training language models to follow instructions with human feedback
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Self-Refine: Iterative Refinement with Self-Feedback
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Training language models to follow instructions with human feedback
Cited by top-tier papers1
Builds on24
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
Related papers
- Rethinking Chain-of-Thought from the Perspective of Self-TrainingZongqian Wu, Baoduo Xu, Ruochen Cui, Mengmeng Zhan et al.ICML 2025
- Large Language Models Can Self-ImproveJiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu et al.EMNLP 2023 · 184 citations
- Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained RewardsHyeonbin Hwang, Doyoung Kim, Seungone Kim, Seonghyeon Ye et al.EMNLP Findings 2024 · 6 citations
- Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled DataKashun Shum, Shizhe Diao, Tong ZhangEMNLP Findings 2023 · 63 citations
- Critique-RL: Training Language Models For Critiquing Through Two-Stage Reinforcement LearningZhiheng Xi, Jixuan Huang, Xin Guo, Boyang Hong et al.ICLR 2026 · 4 citations
