Lune

ACL Findings2026

Automatic Combination of Sample Selection Strategies for Few-Shot Learning

Branislav Pecher, Ivan Srba, Mária Bieliková, Joaquin Vanschoren

Abstract

In few-shot learning, the selection of samples has a significant impact on the performance of the model. While effective sample selection strategies are well-established in supervised settings, research on large language models largely overlooks them, favouring strategies specifically tailored to individual in-context learning settings. In this paper, we propose a new method for Automatic Combination of SamplE Selection Strategies (ACSESS) to leverage the strengths and complementarity of various well-established selection objectives. We investigate and compare the impact of 23 sample selection strategies on the performance of 5 in-context learning models and 3 few-shot learning approaches (meta-learning, few-shot fine-tuning) over 6 text and 8 image datasets. The experimental results show that the combination of strategies through the ACSESS method consistently outperforms all individual selection strategies and performs on par or exceeds the in-context learning specific baselines. Lastly, we demonstrate that sample selection remains effective even on smaller datasets, yielding the greatest benefits when only a few shots are selected, while its advantage diminishes as the number of shots increases.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Deep Learning on a Data Diet: Finding Important Examples Early in Training cites Selection via Proxy: Efficient Data Selection for Deep Learning
  • Deep Learning on a Data Diet: Finding Important Examples Early in Training cites GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning
  • Deep Learning on a Data Diet: Finding Important Examples Early in Training cites Coresets for Data-efficient Training of Machine Learning Models
  • Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performance cites On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices
  • GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning cites Coresets for Data-efficient Training of Machine Learning Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even PerformanceBranislav Pecher, Ivan Srba, Mária BielikováEMNLP 2025 · 6 citations
  2. On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic ChoicesBranislav Pecher, Ivan Srba, Mária BielikováEMNLP 2024 · 1 citation
  3. Difficulty-Diversity Collaborative Filtering for Data-Efficient LLM Fine-TuningLong P. Hoang, Wenxuan Zhang, Wei LuICLR 2026

Builds on33

  1. Deep Learning on a Data Diet: Finding Important Examples Early in TrainingMansheej Paul, Surya Ganguli, Gintare Karolina DziugaiteNeurIPS 2021 · 806 citations
  2. Coresets for Data-efficient Training of Machine Learning ModelsBaharan Mirzasoleiman, Jeff A. Bilmes, Jure LeskovecICML 2020 · 494 citations
  3. Selection via Proxy: Efficient Data Selection for Deep LearningCody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman et al.ICLR 2020 · 462 citations
  4. GLISTER: Generalization based Data Subset Selection for Efficient and Robust LearningKrishnaTeja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, Rishabh K. IyerAAAI 2021 · 300 citations
  5. Retrieval-Based Prompt Selection for Code-Related Few-Shot LearningNoor Nashid, Mifta Sintaha, Ali MesbahICSE 2023 · 156 citations

Related papers

  1. Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and OrderingZhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng KongACL 2023 · 50 citations
  2. Making Large Vision Language Models to Be Good Few-Shot LearnersFan Liu, Wenwen Cai, Jian Huo, Chuanyi Zhang et al.AAAI 2025 · 7 citations
  3. Exploring Diverse In-Context Configurations for Image CaptioningXu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen et al.NeurIPS 2023 · 104 citations
  4. Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text AugmentationJán Cegin, Branislav Pecher, Jakub Simko, Ivan Srba et al.EMNLP Findings 2025
  5. Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot LearningYu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang et al.ICML 2023 · 64 citations
Dusk over the sea between two cliffs drawn in fine vertical lines