ACL Findings2024
Chain-of-Quizzes: Pedagogy-inspired Example Selection in In-Context-Learning
Yiquan Wu, Anlai Zhou, Yuhang Liu, Yifei Liu, Adam Jatowt, Weiming Lu, Jun Xiao, Kun Kuang
Abstract
In-context learning (ICL) has emerged as a powerful tool for enhancing large language models (LLMs) in addressing downstream tasks. In this paper, we explore the vital task of example selection in ICL by mimicking the human learning process. We propose a Chain-of-Quizzes (CoQ) framework inspired by educational theories such as Bruner's Spiral Learning and Mastery Learning theory. Specifically, our framework employs the LLMs to answer the quiz (question in the example) to sift 'good' examples, combines these examples iteratively with the increasing complexity, and utilizes a final exam to gauge the combined example chains. Our extensive experiments on diverse reasoning datasets show the proposed approach outperforms baseline models. These findings underscore the framework's potential for future research. The code and data will be made available here 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Language Models are Few-Shot Learners
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Large Language Models are Zero-Shot Reasoners
- Large Language Models are Zero-Shot Reasoners cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Language Models are Few-Shot Learners
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds on19
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
Related papers
- LaRS: Latent Reasoning Skills for Chain-of-Thought ReasoningZifan Xu, Haozhu Wang, Dmitriy Bespalov, Xian Wu et al.EMNLP Findings 2024 · 4 citations
- Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex ReasoningXuetao Ma, Wenbin Jiang, Hua HuangACL Findings 2025 · 3 citations
- Revisiting Chain-of-Thought Prompting: Zero-shot Can Be Stronger than Few-shotXiang Cheng, Chengyan Pan, Minjun Zhao, Deyang Li et al.EMNLP Findings 2025 · 4 citations
- What Do Language Models Learn in Context? The Structured Task HypothesisJiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan CotterellACL 2024 · 5 citations
- Chain-of-Table: Evolving Tables in the Reasoning Chain for Table UnderstandingZilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos et al.ICLR 2024 · 244 citations
