EMNLP Findings2023
GRACE: Discriminator-Guided Chain-of-Thought Reasoning
Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang
Abstract
In the context of multi-step reasoning, e.g., with chain-of-thought, language models (LMs) can easily assign a high likelihood to incorrect steps. As a result, decoding strategies that optimize for solution likelihood often yield incorrect solutions. To address this issue, we propose Guiding chain-of-thought ReAsoning with a CorrectnEss Discriminator (GRACE), a stepwise decoding approach that steers the decoding process towards producing correct reasoning steps. GRACE employs a discriminator trained with a contrastive loss over correct and incorrect steps, which is used during decoding to score next-step candidates based on their correctness. Importantly, GRACE only requires sampling from the LM, without the need for LM training or fine-tuning. Using models from FLAN-T5 and LLaMA families, we evaluate GRACE over four math and two symbolic reasoning tasks, where it exhibits substantial performance gains compared to greedy decoding, verifiers, and self-consistency in most settings. When further combined with self-consistency, GRACE outperforms all the baselines by sizeable margins. Human and LLM evaluations over GSM8K show that GRACE not only improves the final answer accuracy but also the correctness of the intermediate reasoning. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
16 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models cites Large Language Models are Zero-Shot Reasoners
- mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
- Finetuned Language Models are Zero-Shot Learners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Finetuned Language Models are Zero-Shot Learners cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Finetuned Language Models are Zero-Shot Learners
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Calibrating Reasoning in Language Models with Internal Consistency cites Large Language Models are Zero-Shot Reasoners
- Calibrating Reasoning in Language Models with Internal Consistency cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- ARGS: Alignment as Reward-Guided Search cites Training language models to follow instructions with human feedback
Cited by top-tier papers16
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit DifferenceJiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu et al.NeurIPS 2024 · 106 citations
- ARGS: Alignment as Reward-Guided SearchMaxim Khanov, Jirayu Burapacheep, Yixuan LiICLR 2024 · 101 citations
- Calibrating Reasoning in Language Models with Internal ConsistencyZhihui Xie, Jizhou Guo, Tong Yu, Shuai LiNeurIPS 2024 · 37 citations
- MedAdapter: Efficient Test-Time Adaptation of Large Language Models Towards Medical ReasoningWenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu et al.EMNLP 2024 · 7 citations
- mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language ModelsHuiyuan Lai, Malvina NissimACL 2024 · 5 citations
Builds on18
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
Related papers
- Self-Evaluation Guided Beam Search for ReasoningYuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao et al.NeurIPS 2023 · 316 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Enhancing Mathematical Reasoning in LLMs by Stepwise CorrectionZhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan et al.ACL 2025
- CRANE: Reasoning with constrained LLM generationDebangshu Banerjee, Tarun Suresh, Shubham Ugare, Sasa Misailovic et al.ICML 2025
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMsXueyan Li, Guinan Su, Mrinmaya Sachan, Jonas GeipingICLR 2026 · 5 citations
