ACL Findings2025
Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models?
Yifei Wang, Yu Sheng, Linjing Li, Daniel Dajun Zeng
Abstract
Recent advances in handling long sequences have facilitated the exploration of long-context in-context learning (ICL). While much of the existing research emphasizes performance improvements driven by additional in-context examples, the influence on the trustworthiness of generated responses remains underexplored. This paper addresses this gap by investigating how increased examples influence predictive uncertainty-an essential aspect in trustworthiness. We begin by systematically quantifying the uncertainty of ICL with varying shot counts, analyzing the impact of example quantity. Through uncertainty decomposition, we introduce a novel perspective on performance enhancement, with a focus on epistemic uncertainty (EU). Our results reveal that additional examples reduce total uncertainty in both simple and complex tasks by injecting task-specific knowledge, thereby diminishing EU and enhancing performance. For complex tasks, these advantages emerge only after addressing the increased noise and uncertainty associated with longer inputs. Finally, we explore the evolution of internal confidence across layers, unveiling the mechanisms driving the reduction in uncertainty.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Seeing Is Coding: On the Effectiveness of Vision Language Models in Code Understanding cites FASA: FREQUENCY-AWARE SPARSE ATTENTION
- FASA: FREQUENCY-AWARE SPARSE ATTENTION cites Efficient Streaming Language Models with Attention Sinks
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Cited by top-tier papers2
Builds on22
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- Efficient Streaming Language Models with Attention SinksGuangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han et al.ICLR 2024 · 1,714 citations
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsMaciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger et al.AAAI 2024 · 1,292 citations
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe et al.EMNLP 2022 · 634 citations
Related papers
- Revisiting In-Context Learning with Long Context Language ModelsJinheon Baek, Sun Jae Lee, Prakhar Gupta, Geunseob Oh et al.ACL Findings 2025 · 2 citations
- Variational Uncertainty Decomposition for In-Context LearningI. Shavindra Jayasekera, Jacob Si, Filippo Valdettaro, Wenlong Chen et al.NeurIPS 2025 · 7 citations
- Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction ConfidenceJinseok Chung, Minkyoung Song, Hyunji Jung, Namhoon LeeACL 2026
- IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model GenerationHaozhi Fan, Jinhao Duan, Kaidi XuACL 2026 · 1 citation
- Decomposing Uncertainty for Large Language Models through Input Clarification EnsemblingBairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas et al.ICML 2024 · 113 citations
