ELICIT: LLM Augmentation Via External In-context Capability
Abstract
Enhancing the adaptive capabilities of large language models is a critical pursuit in both research and application. Traditional fine-tuning methods require substantial data and computational resources, especially for enhancing specific capabilities, while in-context learning is limited by the need for appropriate demonstrations and efficient token usage. Inspired by the expression of in-context learned capabilities through task vectors and the concept of modularization, we propose ELICIT, a framework consisting of two modules designed to effectively store and reuse task vectors to elicit the diverse capabilities of models without additional training or inference tokens. Our comprehensive experiments and analysis demonstrate that our pipeline is highly transferable across different input formats, tasks, and model architectures. ELICIT serves as a plug-and-play performance booster to enable adaptive elicitation of model capabilities. By externally storing and reusing vectors that represent in-context learned capabilities, ELICIT not only demonstrates the potential to operate modular capabilities but also significantly enhances the performance, versatility, adaptability, and scalability of large language models. Our code is publicly available 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Enhancing Large Language Models for Time-Series Forecasting via Vector-Injected In-Context Learning cites Language Models are Few-Shot Learners
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling cites Language Models are Few-Shot Learners
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity cites Language Models are Few-Shot Learners
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity cites Calibrate Before Use: Improving Few-shot Performance of Language Models
- ContextNav: Towards Agentic Multimodal In-Context Learning cites Language Models are Few-Shot Learners
- ContextNav: Towards Agentic Multimodal In-Context Learning cites Transformers Learn In-Context by Gradient Descent
- ContextNav: Towards Agentic Multimodal In-Context Learning cites Calibrate Before Use: Improving Few-shot Performance of Language Models
- Transformers Learn In-Context by Gradient Descent cites Language Models are Few-Shot Learners
- Calibrate Before Use: Improving Few-shot Performance of Language Models cites Language Models are Few-Shot Learners
Cited by top-tier papers2
- ContextNav: Towards Agentic Multimodal In-Context LearningHonghao Fu, Yuan Ouyang, Kai-Wei Chang, Yiwei Wang et al.ICLR 2026 · 14 citations
- Enhancing Large Language Models for Time-Series Forecasting via Vector-Injected In-Context LearningJianqi Zhang, Jingyao Wang, Wenwen Qiang, Fanjiang Xu et al.WWW 2026
Builds on34
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein et al.ICML 2021 · 1,843 citations
- Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order SensitivityYao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel et al.ACL 2022 · 1,494 citations
- Transformers Learn In-Context by Gradient DescentJohannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento et al.ICML 2023 · 729 citations
Related papers
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space SteeringSheng Liu, Haotian Ye, Lei Xing, James Y. ZouICML 2024 · 244 citations
- Quantifying Elicitation of Latent Capabilities in Language ModelsElizabeth Donoway, Hailey Joren, Arushi Somani, Henry Sleight et al.NeurIPS 2025 · 4 citations
- Skills-in-Context: Unlocking Compositionality in Large Language ModelsJiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song et al.EMNLP Findings 2024 · 4 citations
- ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool learningXingshan Zeng, Weiwen Liu, Xu Huang, Zezhong Wang et al.AAAI 2026 · 3 citations
- Reusable Experiences: Latent Routing and Modular Composition in LLMsShuai Ling, Lizi Liao, Dongmei Jiang, Weili GuanACL 2026
