EMNLP Findings2025
GRAD: Generative Retrieval-Aligned Demonstration Sampler for Efficient Few-Shot Reasoning
Oussama Gabouj, Kamel Charaf, Ivan Zakazov, Nicolas Mario Baldwin, Robert West
Abstract
Large Language Models (LLMs) achieve strong performance across diverse tasks, but their effectiveness often depends on the quality of the provided context. Retrieval-Augmented Generation (RAG) enriches prompts with external information, but its reliance on static databases constrains adaptability and can result in irrelevant demonstrations. In this work, we propose a Generative Retrieval-Aligned Demonstrator (GRAD), a dynamic demonstration-based approach where an LLM model is trained to generate input-specific concise demonstrations. By tailoring demonstrations to each input, our method offers better contextual support than traditional RAG approaches. We demonstrate the superiority of GRAD under budget constraints, where we limit both the number of tokens used per demonstration and the number of tokens used for the final output. Trained solely on a math dataset, GRAD consistently outperforms strong baselines on Qwen2.5-14B across mathematical reasoning and advanced STEM questions, highlighting GRAD's robust generalization to out-of-distribution (OOD) domains such as physics, chemistry, and computer science. Furthermore, we show that demonstrations generated by trained smaller models can effectively guide larger target models, reducing training costs while maintaining competitive accuracy. Overall, this work introduces a scalable demonstration generator model presenting the first step toward a dynamic few-shot learning paradigm in resource-constrained settings. We release the code used for the project: https://github.com/charafkamel/ GRAD-demonstration-sampler
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Measuring Massive Multitask Language Understanding cites Language Models are Few-Shot Learners
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
- Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds on11
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
Related papers
- Selecting Demonstrations for Many-Shot In-Context Learning via Gradient MatchingJianfei Zhang, Bei Li, Jun Bai, Rumei Li et al.ACL Findings 2025
- AlignedCoT: Prompting Large Language Models via Native-Speaking DemonstrationsZhicheng Yang, Yinya Huang, Jing Xiong, Liang Feng et al.EMNLP Findings 2024 · 1 citation
- PaRaDe: Passage Ranking using Demonstrations with LLMsAndrew Drozdov, Honglei Zhuang, Zhuyun Dai, Zhen Qin et al.EMNLP Findings 2023 · 13 citations
- Meta-training with Demonstration Retrieval for Efficient Few-shot LearningAaron Mueller, Kanika Narang, Lambert Mathias, Qifan Wang et al.ACL Findings 2023
- Promptagator: Few-shot Dense Retrieval From 8 ExamplesZhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan et al.ICLR 2023 · 46 citations
