Towards Sustainable Investment Policies Informed by Opponent Shaping
Juan Agustin Duque, Razvan Ciuca, Ayoub Echchahed, Hugo Larochelle, Aaron Courville
Abstract
Addressing climate change requires global coordination, yet rational economic actors often prioritize immediate gains over collective welfare, resulting in social dilemmas. InvestESG is a recently proposed multi-agent simulation that captures the dynamic interplay between investors and companies under climate risk. We provide a formal characterization of the conditions under which InvestESG exhibits an intertemporal social dilemma, deriving theoretical thresholds at which individual incentives diverge from collective welfare. Building on this, we apply Advantage Alignment, a scalable opponent shaping algorithm shown to be effective in general-sum games, to influence agent learning in InvestESG. We offer theoretical insights into why Advantage Alignment systematically favors socially beneficial equilibria by biasing learning dynamics toward cooperative outcomes. Our results demonstrate that strategically shaping the learning processes of economic agents can result in better outcomes that could inform policy mechanisms to better align market incentives with long-term sustainability goals.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 834acd37-4c63-4749-b192-ffa79f3bb578Builds on7
- COLA: Consistent Learning with Opponent-Learning AwarenessTimon Willi, Alistair Letcher, Johannes Treutlein, Jakob N. FoersterICML 2022 · 61 citations
- Model-Free Opponent ShapingChristopher Lu, Timon Willi, Christian A. Schröder de Witt, Jakob N. FoersterICML 2022 · 53 citations
- Proximal Learning With Opponent-Learning AwarenessStephen Zhao, Chris Lu, Roger B. Grosse, Jakob N. FoersterNeurIPS 2022 · 31 citations
- LOQA: Learning with Opponent Q-Learning AwarenessMilad Aghajohari, Juan Agustin Duque, Tim Cooijmans, Aaron C. CourvilleICLR 2024 · 9 citations
- InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemmaXiaoxuan Hou, Jiayi Yuan, Joel Z. Leibo, Natasha JaquesICLR 2025
Related papers
- Advantage Alignment AlgorithmsJuan Agustin Duque, Milad Aghajohari, Tim Cooijmans, Razvan Ciuca et al.ICLR 2025
- Reciprocal Reward Influence Encourages Cooperation From Self-Interested AgentsJohn L. Zhou, Weizhe Hong, Jonathan C. KaoNeurIPS 2024 · 5 citations
- Learning to Incentivize Other Learning AgentsJiachen Yang, Ang Li, Mehrdad Farajtabar, Peter Sunehag et al.NeurIPS 2020 · 105 citations
- Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game PerspectiveHaichuan Wang, Tao Lin, Lingkai Kong, Ce Li et al.ICML 2026 · 3 citations
- Inducing Equilibria via Incentives: Simultaneous Design-and-Play Ensures Global ConvergenceBoyi Liu, Jiayang Li, Zhuoran Yang, Hoi-To Wai et al.NeurIPS 2022 · 29 citations
