EMNLP Findings2024
In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Models
Pengrui Han, Peiyang Song, Haofei Yu, Jiaxuan You
Abstract
Recent advancements in artificial intelligence have led to the creation of highly capable large language models (LLMs) that can perform tasks in a human-like manner. However, LLMs exhibit only infant-level cognitive abilities in certain areas. One such area is the A-Not-B error, a phenomenon seen in infants where they repeat a previously rewarded behavior despite well-observed changed conditions. This highlights their lack of inhibitory control -the ability to stop a habitual or impulsive response. In our work, we design a text-based multi-choice QA scenario similar to the A-Not-B experimental settings to systematically test the inhibitory control abilities of LLMs. We found that stateof-the-art LLMs (like Llama3-8b) perform consistently well with in-context learning (ICL) but make errors and show a significant drop of as many as 83.3% in reasoning tasks when the context changes trivially. This suggests that LLMs only have inhibitory control abilities on par with human infants in this regard, often failing to suppress the previously established response pattern during ICL.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- STaR: Bootstrapping Reasoning With Reasoning cites An Explanation of In-context Learning as Implicit Bayesian Inference
- STaR: Bootstrapping Reasoning With Reasoning cites Language Models are Few-Shot Learners
- STaR: Bootstrapping Reasoning With Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs cites Language Models are Few-Shot Learners
- seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- An Explanation of In-context Learning as Implicit Bayesian Inference cites Language Models are Few-Shot Learners
- Language Models are Few-Shot Learners cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Cited by top-tier papers3
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic CorpusTerufumi Morishita, Gaku Morio, Atsuki Yamaguchi, Yasuhiro SogawaNeurIPS 2024 · 60 citations
- The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMsPengrui Han, Rafal Kocielnik, Peiyang Song, Ramit Debnath et al.ICML 2026 · 33 citations
- seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMsMohammad Ramezanali, Mo Vazifeh, Paolo SantiEMNLP 2025
Builds on24
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
- STaR: Bootstrapping Reasoning With ReasoningEric Zelikman, Yuhuai Wu, Jesse Mu, Noah D. GoodmanNeurIPS 2022 · 1,126 citations
- An Explanation of In-context Learning as Implicit Bayesian InferenceSang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu MaICLR 2022 · 1,030 citations
Related papers
- What Do Language Models Learn in Context? The Structured Task HypothesisJiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan CotterellACL 2024 · 5 citations
- Conditional and Modal Reasoning in Large Language ModelsWesley H. Holliday, Matthew Mandelkern, Cedegao ZhangEMNLP 2024 · 5 citations
- Task Descriptors Help Transformers Learn Linear Models In-ContextRuomin Huang, Rong GeICLR 2025
- Do Large Language Models Know What They Are Capable Of?Casey O. Barkan, Sidney Black, Oliver SourbutICLR 2026 · 11 citations
- Why Larger Language Models Do In-context Learning Differently?Zhenmei Shi, Junyi Wei, Zhuoyan Xu, Yingyu LiangICML 2024 · 54 citations
