Lune

EMNLP Findings2024

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment

Heyan Huang, Yinghao Li, Huashan Sun, Yu Bai, Yang Gao

Abstract

Recent studies have demonstrated that In-Context Learning (ICL), through the use of specific demonstrations, can align Large Language Models (LLMs) with human preferences known as In-Context Alignment (ICA), indicating that models can comprehend human instructions without requiring parameter adjustments. However, the exploration of the mechanism and applicability of ICA remains limited. In this paper, we begin by dividing the context text used in ICA into three categories: format, system prompt, and example. Through ablation experiments, we investigate the effectiveness of each part in enabling ICA to function effectively. We then examine how variants in these parts impact the model's alignment performance. Our findings indicate that the example part is crucial for enhancing the model's alignment capabilities, with changes in examples significantly affecting alignment performance. We also conduct a comprehensive evaluation of ICA's zero-shot capabilities in various alignment tasks. The results indicate that compared to parameter fine-tuning methods, ICA demonstrates superior performance in knowledge-based tasks and tool-use tasks. However, it still exhibits certain limitations in areas such as multi-turn dialogues and instruction following. Source codes and scripts are available at https://github.com/ li-aolong/how-far-can-ica-go.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Reasoning Shapes Alignment: Investigating Cultural Alignment in Large Reasoning Models with Cultural Norms cites LoRA: Low-Rank Adaptation of Large Language Models
  • QLoRA: Efficient Finetuning of Quantized LLMs cites LoRA: Low-Rank Adaptation of Large Language Models
  • Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements cites LoRA: Low-Rank Adaptation of Large Language Models
  • Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements cites The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
  • AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference cites The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
  • Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers cites Transformers Learn In-Context by Gradient Descent
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value DifferenceJing Yao, Shitong Duan, Xiaoyuan Yi, Dongkuan Xu et al.ICLR 2026 · 4 citations
  2. Reasoning Shapes Alignment: Investigating Cultural Alignment in Large Reasoning Models with Cultural NormsYuhang Wang, Yanxu Zhu, Jitao SangAAAI 2026
  3. Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety RequirementsJingyu Zhang, Ahmed Elgohary, Ahmed Magooda, Daniel Khashabi et al.ICLR 2025

Builds on15

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  3. Transformers Learn In-Context by Gradient DescentJohannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento et al.ICML 2023 · 729 citations
  4. The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context LearningBill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri et al.ICLR 2024 · 299 citations
  5. Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-OptimizersDamai Dai, Yutao Sun, Li Dong, Yaru Hao et al.ACL Findings 2023 · 154 citations

Related papers

  1. What Do Language Models Learn in Context? The Structured Task HypothesisJiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan CotterellACL 2024 · 5 citations
  2. Customizing In-context Learning for Dynamic Interest Adaption in LLM-based RecommendationKeqin Bao, Ming Yan, Yang Zhang, Jizhi Zhang et al.ACL Findings 2025 · 1 citation
  3. Task Descriptors Help Transformers Learn Linear Models In-ContextRuomin Huang, Rong GeICLR 2025
  4. The Missing Alignment Link of In-context Learning on SequencesHarshvardhan Agarwal, Sunita SarawagiICML 2025
  5. Take Off the Training Wheels! Progressive In-Context Learning for Effective AlignmentZhenyu Liu, Dongfang Li, Xinshuo Hu, Xinping Zhao et al.EMNLP 2024 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines