Task Descriptors Help Transformers Learn Linear Models In-Context

Large language models (LLM) exhibit strong in-context learning (ICL) ability, which allows the model to make predictions on new examples based on the given prompt. Recently, a line of research (