EMNLP Findings2025
OptiSeq: Ordering Examples On-The-Fly for In-Context Learning
Rahul Atul Bhope, Praveen Venkateswaran, K. R. Jayaram, Vatche Isahagian, Vinod Muthusamy, Nalini Venkatasubramanian
Abstract
Developers using LLMs and LLM-based agents in their applications have provided plenty of anecdotal evidence that in-contextlearning (ICL) is fragile.In this paper, we show that in addition to the quantity and quality of examples, the order in which the incontext examples are listed in the prompt affects the output of the LLM and, consequently, their performance.While prior work has explored improving ICL through datasetdependent techniques, we introduce OptiSeq, a purely inference-time, dataset-free optimization method that efficiently determines the best example order.OptiSeq leverages log probabilities of LLM-generated outputs to systematically prune the search space of possible orderings and recommend the best order(s) by distinguishing orderings that yield high levels of accuracy and those that underperform.Extensive empirical evaluation on multiple LLMs, datasets, and prompts demonstrates that Op-tiSeq improves accuracy by 5.5 -10.5 percentage points across multiple tasks. Utterance Generated Sequence Order Precision Recall Accuracy Log Prob OptiSeqUtterance: I need to retrieve the pending messages from my device with ID 123456.Please provide the pending messages using
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity cites Language Models are Few-Shot Learners
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity cites Calibrate Before Use: Improving Few-shot Performance of Language Models
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts cites Language Models are Few-Shot Learners
- AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse cites Language Models are Few-Shot Learners
- Calibrate Before Use: Improving Few-shot Performance of Language Models cites Language Models are Few-Shot Learners
Cited by top-tier papers2
- TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence ConfigurationYanshu Li, Jianjiang Yang, Tian Yun, Pinyuan Feng et al.EMNLP 2025 · 2 citations
- AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache ReuseJie Ou, Jinyu Guo, Shiyao Guo, Yuang Li et al.ACL 2026
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein et al.ICML 2021 · 1,843 citations
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order SensitivityYao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel et al.ACL 2022 · 1,494 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
Related papers
- Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of ExemplarsZhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, Wenyang Hu et al.NeurIPS 2024 · 44 citations
- Where to show Demos in Your Prompt: A Positional Bias of In-Context LearningKwesi A. Cobbina, Tianyi ZhouEMNLP 2025
- Se²: Sequential Example Selection for In-Context LearningHaoyu Liu, Jianfeng Liu, Shaohan Huang, Yuefeng Zhan et al.ACL Findings 2024 · 3 citations
- What Makes a Good Order of Examples in In-Context LearningQi Guo, Leiyu Wang, Yidong Wang, Wei Ye et al.ACL Findings 2024 · 8 citations
- SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt OptimizationWendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun et al.ACL 2025
