On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning

Multi-agent Inverse Reinforcement Learning (MAIRL) aims to recover agent reward functions from expert demonstrations. We characterize the feasible reward set in Markov games, identifying all reward functions that rationalize a given equilibrium. However, equilibrium-based observations are often ambiguous: a single Nash equilibrium can correspond to many reward structures, potentially changing the game's nature in multi-agent systems. We address this by introducing entropyregularized Markov games, which yield a unique equilibrium while preserving strategic incentives. For this setting, we provide a sample complexity analysis detailing how errors affect learned policy performance. Our work establishes theoretical foundations and practical insights for MAIRL.

We present the essential background and notation used throughout this paper, also summarized in Appendix A.

Let X be a finite set, then we denote by R X all functions mapping from X to R. Additionally, we denote by ∆ X the set of probability measures over X . For n ∈ N we use [n] := 1, . . . , n. We introduce for a (pre)metric space (X , d) with Y, Y ′ ⊆ X