Real World Games Look Like Spinning Tops
Wojciech M. Czarnecki, Gauthier Gidel, Brendan D. Tracey, Karl Tuyls, Shayegan Omidshafiei, David Balduzzi, Max Jaderberg
摘要
This paper investigates the geometrical properties of real world games (e.g. Tic-Tac-Toe, Go, StarCraft II). We hypothesise that their geometrical structure resembles a spinning top, with the upright axis representing transitive strength, and the radial axis representing the non-transitive dimension, which corresponds to the number of cycles that exist at a particular transitive strength. We prove the existence of this geometry for a wide class of real world games by exposing their temporal nature. Additionally, we show that this unique structure also has consequences for learning -it clarifies why populations of strategies are necessary for training of agents, and how population size relates to the structure of the game. Finally, we empirically validate these claims by using a selection of nine real world two-player zero-sum symmetric games, showing 1) the spinning top structure is revealed and can be easily reconstructed by using a new method of Nash clustering to measure the interaction between transitive and cyclical strategy behaviour, and 2) the effect that population size has on the convergence of learning in these games.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper22
- Language Agents with Reinforcement Learning for Strategic Play in the Werewolf GameZelai Xu, Chao Yu, Fei Fang, Yu Wang 等ICML 2024 · 被引用 145 次
- Towards Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum GamesXiangyu Liu, Hangtian Jia, Ying Wen, Yujing Hu 等NeurIPS 2021 · 被引用 67 次
- Adversarial Policies Beat Superhuman Go AIsTony Tong Wang, Adam Gleave, Tom Tseng, Kellin Pelrine 等ICML 2023 · 被引用 34 次
- Policy Space Diversity for Non-Transitive GamesJian Yao, Weiming Liu, Haobo Fu, Yaodong Yang 等NeurIPS 2023 · 被引用 28 次
- NeuPL: Neural Population LearningSiqi Liu, Luke Marris, Daniel Hennes, Josh Merel 等ICLR 2022 · 被引用 19 次
相关 Paper
- Convergence of No-Swap-Regret Dynamics in Self-PlayRenato Paes Leme, Georgios Piliouras, Jon SchneiderNeurIPS 2024 · 被引用 3 次
- Synchronization in Learning in Periodic Zero-Sum Games Triggers Divergence from Nash EquilibriumYuma Fujimoto, Kaito Ariu, Kenshi AbeAAAI 2025 · 被引用 4 次
- Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum GamesStratis Skoulakis, Tanner Fiez, Ryann Sim, Georgios Piliouras 等AAAI 2021 · 被引用 17 次
- Implicit Learning Dynamics in Stackelberg Games: Equilibria Characterization, Convergence Analysis, and Empirical StudyTanner Fiez, Benjamin Chasnov, Lillian J. RatliffICML 2020 · 被引用 144 次
- Follow-the-Regularized-Leader Routes to Chaos in Routing GamesJakub Bielawski, Thiparat Chotibut, Fryderyk Falniowski, Grzegorz Kosiorowski 等ICML 2021 · 被引用 29 次
