Lune

ACL Findings2025

Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic

Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun

Abstract

Self-critic has become a crucial mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts for intuitive instance-level feedback, which resembles System-1 processes and limits the reasoning capabilities. Moreover, there is a lack of in-depth investigations into the relationship between LLM's ability to criticize and its task-solving performance. To address these issues, we propose Critic-CoT, a novel framework that pushes LLMs toward System-2-like critic capability. Through a step-wise CoT reasoning paradigm and the automatic construction of weak-supervision data without human annotation, Critic-CoT enables LLMs to engage in slow, analytic self-critique and refinement, thereby improving their reasoning abilities. Experiments on GSM8K and MATH and out-of-domain evaluation demonstrate that our enhanced model significantly boosts tasksolving performance by filtering out invalid solutions or iterative refinement. Furthermore, we investigate the intrinsic correlation between critique and task-solving abilities within LLMs, discovering that these abilities can mutually reinforce each other rather than conflict. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Self-Refine: Iterative Refinement with Self-Feedback cites Training language models to follow instructions with human feedback
  • Reflexion: language agents with verbal reinforcement learning cites Self-Refine: Iterative Refinement with Self-Feedback
  • Let's Verify Step by Step cites Training language models to follow instructions with human feedback
  • Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Self-Refine: Iterative Refinement with Self-Feedback
  • Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. RLSeek: Evidence-Grounded Reasoning for RAG Hallucination DetectionZhaoheng Huang, Dacheng Wen, Yutao Zhu, Xiaoying Lian et al.ACL 2026

Builds on24

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  3. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  4. Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
  5. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations

Related papers

  1. Rethinking Chain-of-Thought from the Perspective of Self-TrainingZongqian Wu, Baoduo Xu, Ruochen Cui, Mengmeng Zhan et al.ICML 2025
  2. Large Language Models Can Self-ImproveJiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu et al.EMNLP 2023 · 184 citations
  3. Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained RewardsHyeonbin Hwang, Doyoung Kim, Seungone Kim, Seonghyeon Ye et al.EMNLP Findings 2024 · 6 citations
  4. Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled DataKashun Shum, Shizhe Diao, Tong ZhangEMNLP Findings 2023 · 63 citations
  5. Critique-RL: Training Language Models For Critiquing Through Two-Stage Reinforcement LearningZhiheng Xi, Jixuan Huang, Xin Guo, Boyang Hong et al.ICLR 2026 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines