ACL Findings2025
Ranked Voting based Self-Consistency of Large Language Models
Weiqin Wang, Yile Wang, Hui Huang
Abstract
Majority voting is considered an effective method to enhance chain-of-thought reasoning, as it selects the answer with the highest "self-consistency" among different reasoning paths (Wang et al., 2023). However, previous chain-of-thought reasoning methods typically generate only a single answer in each trial, thereby ignoring the possibility of other potential answers. As a result, these alternative answers are often overlooked in subsequent voting processes. In this work, we propose to generate ranked answers in each reasoning process and conduct ranked voting among multiple ranked answers from different responses, thereby making the overall self-consistency more reliable. Specifically, we use three ranked voting methods: Instant-runoff voting, Borda count voting, and mean reciprocal rank voting. We validate our methods on six datasets, including three multiple-choice and three openended question-answering tasks, using both advanced open-source and closed-source large language models. Extensive experimental results indicate that our proposed method outperforms the baselines, showcasing the potential of leveraging the information of ranked answers and using ranked voting to improve reasoning performance. The code is available at https: //github.com/szu-tera/RankedVotingSC.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
9 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Optimal Self-Consistency for Efficient Reasoning with Large Language Models cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Optimal Self-Consistency for Efficient Reasoning with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models cites Large Language Models are Zero-Shot Reasoners
- Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learning cites Deep Think with Confidence
- Deep Think with Confidence cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers9
- Deep Think with ConfidenceYichao Fu, Xuewei Wang, Hao Zhang, Yuandong Tian et al.ICLR 2026 · 171 citations
- Efficient Test-Time Scaling for Small Vision-Language ModelsMehmet Onurcan Kaya, Desmond Elliott, Dim PapadopoulosICLR 2026 · 13 citations
- Optimal Self-Consistency for Efficient Reasoning with Large Language ModelsAustin Feng, Marius Alonso, Ambroise Odonnat, Vasilii Feofanov et al.ICML 2026 · 6 citations
- Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement LearningWeiqin Wang, Yile Wang, Kehao Chen, Hui HuangACL 2026 · 5 citations
- Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language ModelsBoxuan Wang, Zhuoyun Li, Xinmiao Huang, Xiaowei Huang et al.ACL 2026 · 2 citations
Builds on12
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsMaciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger et al.AAAI 2024 · 1,292 citations
Related papers
- Answering Questions by Meta-Reasoning over Multiple Chains of ThoughtOri Yoran, Tomer Wolfson, Ben Bogin, Uri Katz et al.EMNLP 2023 · 30 citations
- GCoT-Decoding: Unlocking Deep Reasoning Paths for Universal Question AnsweringGuanran Luo, Wentao Qiu, Zhongquan Jian, Meihong Wang et al.ACL Findings 2026
- DCMKC: A Dual Consistency Matching Approach for Multi-hop Question Answering in LLMsXinyi Wang, Yiping Song, Chang Liu, Tingjin Luo et al.EMNLP Findings 2025
- Diversity Matters: Revisiting Test-Time Compute in Vision-Language ModelsYijie Tong, Yifan Hou, Shaobo Cui, Antoine Bosselut et al.ICML 2026 · 1 citation
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
