Robust agents learn causal world models
Jonathan Richens, Tom Everitt
Abstract
It has long been hypothesised that causal reasoning plays a fundamental role in robust and general intelligence. However, it is not known if agents must learn causal models in order to generalise to new domains, or if other inductive biases are sufficient. We answer this question, showing that any agent capable of satisfying a regret bound under a large set of distributional shifts must have learned an approximate causal model of the data generating process, which converges to the true causal model for optimal agents. We discuss the implications of this result for several research areas including transfer learning and causal inference.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext b76aba44-6536-4975-9307-2af9ebe56a4bCited by top-tier papers23
- Honesty Is the Best Policy: Defining and Mitigating AI DeceptionFrancis Ward, Francesca Toni, Francesco Belardinelli, Tom EverittNeurIPS 2023 · 60 citations
- SPARTAN: A Sparse Transformer World Model Attending to What MattersAnson Lei, Bernhard Schölkopf, Ingmar PosnerNeurIPS 2025 · 12 citations
- Measuring Goal-DirectednessMatt MacDermott, James Fox, Francesco Belardinelli, Tom EverittNeurIPS 2024 · 9 citations
- Agents Robust to Distribution Shifts Learn Causal World Models Even Under MediationMatteo Ceriscioli, Karthika MohanNeurIPS 2025 · 6 citations
- A Principle of Targeted Intervention for Multi-Agent Reinforcement LearningAnjie Liu, Jianhong Wang, Samuel Kaski, Jun Wang et al.NeurIPS 2025 · 3 citations
Builds on8
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Language Models Represent Space and TimeWes Gurnee, Max TegmarkICLR 2024 · 303 citations
- A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate ExperimentsJuan D. Correa, Elias BareinboimAAAI 2020 · 90 citations
- Agent Incentives: A Causal PerspectiveTom Everitt, Ryan Carey, Eric D. Langlois, Pedro A. Ortega et al.AAAI 2021 · 66 citations
- Honesty Is the Best Policy: Defining and Mitigating AI DeceptionFrancis Ward, Francesca Toni, Francesco Belardinelli, Tom EverittNeurIPS 2023 · 60 citations
Related papers
- Theory-Based Causal Transfer: Integrating Instance-Level Induction and Abstract-Level Structure LearningMark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu et al.AAAI 2020 · 28 citations
- Transportability for Bandits with Data from Different EnvironmentsAlexis Bellot, Alan Malek, Silvia ChiappaNeurIPS 2023 · 11 citations
- General agents need world modelsJonathan Richens, Tom Everitt, David AbelICML 2025
- Generative Causal Representation Learning for Out-of-Distribution Motion ForecastingShayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, Mark CrowleyICML 2023 · 21 citations
- Counterfactual Structural Causal BanditsMin Woo Park, Sanghack LeeICLR 2026 · 1 citation
