ACL Findings2024
Latent Learningscape Guided In-context Learning
Anlai Zhou, Sunshine Jiang, Yifei Liu, Yiquan Wu, Kun Kuang, Jun Xiao
Abstract
The growing interest in leveraging large language models is driven by their exceptional imitation and reasoning capabilities.In-context learning (ICL), a streamlined method, has shown potential in boosting these models' performance without modifying their underlying parameters, especially when supplied with suitable demonstrations.However, existing methods mainly choose demonstrations by comparing surface-level semantic similarities (e.g., based on embedding) and fall short of identifying the most fitting ones.This paper introduces the concept of a "latent learningscape", a more nuanced representation that describes the characteristic of the demonstrations.Building on this concept, we develop a results-driven approach to characterize the latent learningscape features of demonstrations, which then inform the creation of more effective prompts.Through comprehensive testing across datasets in arithmetic, commonsense, and symbolic reasoning tasks, our approach outperforms leading models, showing an average increase in scores by 7.4 percentage points.The code and data will be made available here 1 . 1 https://github.com/anlaiJoe/Latent-LearningscapeQ1: In the parking lot, there are 3 Audi A4s and 4 Audi Q5s parked.At this time, 3 Audi Q5s left, and 5 Audi A7s arrived.How many sedans are currently in the parking lot?A1: Initially, there are: 3 Audi A4s (sedans) and 4 Audi Q5s (SUVs) …… Now, let's count the total number of sedans: 3 Audi A4s + 5 Audi A7s = 8 sedans Q2: The number of rabbits in the cage is three times the number of chickens.The number of chickens is half the number of ducks, and there are 6 ducks.How many feet do they have in total?A2: Given: The number of rabbits (R) is three times the number of chickens (C): R=3C.…… Substitute the values: Total feet=36+6+12=54 X: In the cage, there are 5 rabbits, 3 chickens, and after 3 rabbits left, 6 ducks arrived.How many poultry are there in the cage?Embedding Corr.with X Q1 0.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- STaR: Bootstrapping Reasoning With Reasoning cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- STaR: Bootstrapping Reasoning With Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Large Language Models are Zero-Shot Reasoners
- Large Language Models are Zero-Shot Reasoners cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites STaR: Bootstrapping Reasoning With Reasoning
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds on21
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- STaR: Bootstrapping Reasoning With ReasoningEric Zelikman, Yuhuai Wu, Jesse Mu, Noah D. GoodmanNeurIPS 2022 · 1,126 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Least-to-Most Prompting Enables Complex Reasoning in Large Language ModelsDenny Zhou, Nathanael Schärli, Le Hou, Jason Wei et al.ICLR 2023 · 318 citations
Related papers
- Compositional Exemplars for In-context LearningJiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu et al.ICML 2023 · 188 citations
- Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex ReasoningXuetao Ma, Wenbin Jiang, Hua HuangACL Findings 2025 · 3 citations
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe et al.EMNLP 2022 · 634 citations
- Revisiting Demonstration Selection Strategies in In-Context LearningKeqin Peng, Liang Ding, Yancheng Yuan, Xuebo Liu et al.ACL 2024
- Rethinking Label Consistency of In-Context Learning: An Implicit Transductive Label Propagation PerspectiveHaoyang Chen, Richong Zhang, Junfan ChenAAAI 2026
