Task Descriptors Help Transformers Learn Linear Models In-Context

Ruomin Huang; Rong Ge

ICLR2025顶会

Task Descriptors Help Transformers Learn Linear Models In-Context

Ruomin Huang, Rong Ge

出版方

2025年份

1顶会引用

摘要

Large language models (LLM) exhibit strong in-context learning (ICL) ability, which allows the model to make predictions on new examples based on the given prompt. Recently, a line of research (

问问这篇 Paper

智能体会读完全文。

Lune 把这篇 Paper 索引到了每一个公式，引用它的顶会 Paper 也一样。你提问，回答直接引用原文。

核心论断
谁在沿用
横向对比
局限

你的问题

智能体调用

Luneget_paper_fulltext

在 Lune 里问

免费开始，无需绑卡

引用它的顶会 Paper1

Provable In-Context Vector Arithmetic via Retrieving Task ConceptsDake Bu, Wei Huang, Andi Han, Atsushi Nitanda 等ICML 2025

问问它们各自怎么用它

它引用的顶会 Paper7

Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
What Can Transformers Learn In-Context? A Case Study of Simple Function ClassesShivam Garg, Dimitris Tsipras, Percy Liang, Gregory ValiantNeurIPS 2022 · 被引用 883 次
Transformers Learn In-Context by Gradient DescentJohannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento 等ICML 2023 · 被引用 729 次
Transformers learn to implement preconditioned gradient descent for in-context learningKwangjun Ahn, Xiang Cheng, Hadi Daneshmand, Suvrit SraNeurIPS 2023 · 被引用 324 次
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-AttentionArvind V. Mahankali, Tatsunori Hashimoto, Tengyu MaICLR 2024 · 被引用 160 次

摘要

问问这篇 Paper

智能体会读完全文。

引用它的顶会 Paper1

它引用的顶会 Paper7

相关 Paper