Towards Sustainable Investment Policies Informed by Opponent Shaping
Juan Agustin Duque, Razvan Ciuca, Ayoub Echchahed, Hugo Larochelle, Aaron Courville
摘要
Addressing climate change requires global coordination, yet rational economic actors often prioritize immediate gains over collective welfare, resulting in social dilemmas. InvestESG is a recently proposed multi-agent simulation that captures the dynamic interplay between investors and companies under climate risk. We provide a formal characterization of the conditions under which InvestESG exhibits an intertemporal social dilemma, deriving theoretical thresholds at which individual incentives diverge from collective welfare. Building on this, we apply Advantage Alignment, a scalable opponent shaping algorithm shown to be effective in general-sum games, to influence agent learning in InvestESG. We offer theoretical insights into why Advantage Alignment systematically favors socially beneficial equilibria by biasing learning dynamics toward cooperative outcomes. Our results demonstrate that strategically shaping the learning processes of economic agents can result in better outcomes that could inform policy mechanisms to better align market incentives with long-term sustainability goals.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper7
- COLA: Consistent Learning with Opponent-Learning AwarenessTimon Willi, Alistair Letcher, Johannes Treutlein, Jakob N. FoersterICML 2022 · 被引用 61 次
- Model-Free Opponent ShapingChristopher Lu, Timon Willi, Christian A. Schröder de Witt, Jakob N. FoersterICML 2022 · 被引用 53 次
- Proximal Learning With Opponent-Learning AwarenessStephen Zhao, Chris Lu, Roger B. Grosse, Jakob N. FoersterNeurIPS 2022 · 被引用 31 次
- LOQA: Learning with Opponent Q-Learning AwarenessMilad Aghajohari, Juan Agustin Duque, Tim Cooijmans, Aaron C. CourvilleICLR 2024 · 被引用 9 次
- InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemmaXiaoxuan Hou, Jiayi Yuan, Joel Z. Leibo, Natasha JaquesICLR 2025
相关 Paper
- Advantage Alignment AlgorithmsJuan Agustin Duque, Milad Aghajohari, Tim Cooijmans, Razvan Ciuca 等ICLR 2025
- Reciprocal Reward Influence Encourages Cooperation From Self-Interested AgentsJohn L. Zhou, Weizhe Hong, Jonathan C. KaoNeurIPS 2024 · 被引用 5 次
- Learning to Incentivize Other Learning AgentsJiachen Yang, Ang Li, Mehrdad Farajtabar, Peter Sunehag 等NeurIPS 2020 · 被引用 105 次
- Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game PerspectiveHaichuan Wang, Tao Lin, Lingkai Kong, Ce Li 等ICML 2026 · 被引用 3 次
- Inducing Equilibria via Incentives: Simultaneous Design-and-Play Ensures Global ConvergenceBoyi Liu, Jiayang Li, Zhuoran Yang, Hoi-To Wai 等NeurIPS 2022 · 被引用 29 次
