Lune

ACL Findings2025

Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation

Yiwei Li, Ji Zhang, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Jiayi Shi, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li

Abstract

Self-consistency improves reasoning by aggregating diverse stochastic samples, yet the dynamics behind its efficacy remain underexplored. We reframe self-consistency as a dynamic distributional alignment problem, revealing that decoding temperature not only governs sampling randomness but also actively shapes the latent answer distribution. Given that high temperatures require prohibitively large sample sizes to stabilize, while low temperatures risk amplifying biases, we propose a confidence-driven mechanism that dynamically calibrates temperature: sharpening the sampling distribution under uncertainty to align with high-probability modes, and promoting exploration when confidence is high. Experiments on mathematical reasoning tasks show this approach outperforms fixed-diversity baselines under limited samples, improving both average and best-case performance across varying initial temperatures without additional data or modules. This establishes self-consistency as a synchronization challenge between sampling dynamics and evolving answer distributions.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Linking Perception, Confidence and Accuracy in MLLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Making Language Models Better Reasoners with Step-Aware Verifier cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Making Language Models Better Reasoners with Step-Aware Verifier cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Calibrating Large Language Models with Sample Consistency cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Calibrating Large Language Models with Sample Consistency cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Optimal Self-Consistency for Efficient Reasoning with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Optimal Self-Consistency for Efficient Reasoning with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. The Unreasonable Effectiveness of Entropy Minimization in LLM ReasoningShivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han et al.NeurIPS 2025 · 185 citations
  2. Optimal Self-Consistency for Efficient Reasoning with Large Language ModelsAustin Feng, Marius Alonso, Ambroise Odonnat, Vasilii Feofanov et al.ICML 2026 · 6 citations
  3. Linking Perception, Confidence and Accuracy in MLLMsYuetian Du, Yucheng Wang, Rongyu Zhang, Zhijie Xu et al.CVPR 2026 · 2 citations
  4. When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM JudgesSichu Liang, Zhenglin Wang, Jiajia Chu, Pengfei Xia et al.ACL 2026

Builds on10

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  3. Calibrating Large Language Models with Sample ConsistencyQing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang et al.AAAI 2025 · 72 citations
  4. Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language ModelsYuqi Zhu, Jia Li, Ge Li, Yunfei Zhao et al.AAAI 2024 · 68 citations
  5. Making Language Models Better Reasoners with Step-Aware VerifierYifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu et al.ACL 2023 · 52 citations

Related papers

  1. Mirror-Consistency: Harnessing Inconsistency in Majority VotingSiyuan Huang, Zhiyuan Ma, Jintao Du, Changhua Meng et al.EMNLP Findings 2024 · 1 citation
  2. Speculative Decoding for Multi-Sample InferenceYiwei Li, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan et al.EMNLP Findings 2025 · 1 citation
  3. Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMsXueyan Li, Guinan Su, Mrinmaya Sachan, Jonas GeipingICLR 2026 · 5 citations
  4. Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language ModelsWenqing Chen, Weicheng Wang, Zhixuan Chu, Kui Ren et al.ACL Findings 2024 · 5 citations
  5. Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All ScenariosLei Lin, Jia-Yi Fu, Pengli Liu, Qingyang Li et al.ACL Findings 2024 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines