ACL Findings2026
Divergent Thinking: Escape the Homogeneity Trap in Generative Commonsense Reasoning
Yiheng Tao, Kaiwen Cheng, Zhiwei Nie, Chang Liu, Jie Chen
Abstract
Generative commonsense reasoning (GCR) requires models to synthesize coherent narratives that simultaneously satisfy lexical constraints and commonsense logic. Although ensemblebased LLM strategies are widely adopted to alleviate the fragility of single-chain reasoning, we uncover a counterintuitive homogeneity trap in GCR. Specifically, we observe that increasing the number of reasoning chains can degrade performance, as the generated chains tend to collapse into a narrow semantic region, thereby reinforcing shared biases rather than providing complementary evidence. We posit that escaping this trap requires fundamentally broadening semantic coverage via heterogeneous sources. Our investigation into the nature of diversity reveals that deep semantic diversity, rather than surface-level lexical variation, is the decisive prerequisite for effective integration. Motivated by this insight, we propose an Explore-then-Integrate framework, in which high-semanticentropy explorers capture diverse concept bindings, and a powerful integrator performs compositional synthesis to merge valid fragments into coherent narratives. Crucially, to ensure that the observed performance gains arise from accurate logical composition rather than trivial best-candidate selection, we introduce a provenance-aware evaluation suite that explicitly quantifies the heterogeneous origins of synthesized outputs. Extensive experiments on multiple benchmarks demonstrate the consistent superiority of our approach across a range of metrics. Notably, our method achieves over 10% improvement in overall accuracy on NoRa and in SPICE score on CommonGen-Lite.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Large Language Models are Zero-Shot Reasoners
- Large Language Models are Zero-Shot Reasoners cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds on26
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraintWei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang et al.ICML 2024 · 346 citations
- Least-to-Most Prompting Enables Complex Reasoning in Large Language ModelsDenny Zhou, Nathanael Schärli, Le Hou, Jason Wei et al.ICLR 2023 · 318 citations
Related papers
- Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph ExpertsWenhao Yu, Chenguang Zhu, Lianhui Qin, Zhihan Zhang et al.ACL Findings 2022 · 21 citations
- Multi-LLM Collaborative Search for Complex Problem SolvingSen Yang, Yafu Li, Wai Lam, Yu ChengACL Findings 2026 · 1 citation
- KGR4: Retrieval, Retrospect, Refine and Rethink for Commonsense GenerationXin Liu, Dayiheng Liu, Baosong Yang, Haibo Zhang et al.AAAI 2022 · 9 citations
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich SeekingZhengwei Tao, Haiyang SHEN, Baixuan Li, Wenbiao Yin et al.ICLR 2026 · 14 citations
- Balancing Act: Diversity and Consistency in Large Language Model EnsemblesAhmed Abdulaal, Chen Jin, Nina Montaña Brown, Aryo Pradipta Gema et al.ICLR 2025
