Deep Reinforcement Learning Finds Bayes-Nash Equilibrium in Competitive Newsvendor Problems

We investigate learning dynamics in competitive newsvendor games, a class of continuousaction games with strategic substitutes. Despite established equilibrium properties, convergence of independent learning algorithms in repeated general-sum play remains uncertain. We analyze structural properties under complete and incomplete information, deriving closed-form equilibria for a symmetric complete-information benchmark with perfect substitution. Our main theoretical contribution proves strict monotonicity in both complete-information and Bayesian models with private costs, ensuring equilibrium uniqueness. This provides convergence guarantees for variational-inequality-based algorithms. Numerical experiments using deep reinforcement learning agents with Proximal Policy Optimization empirically demonstrate convergence to Nash and Bayesian Nash equilibria, verified by equilibrium checks. These results establish a foundation for applying deep reinforcement learning in competitive inventory management.

We analyze both a complete-information setting, in which agents' cost parameters are fixed, and a Bayesian setting with private cost information. The complete-information model admits a closed-form symmetric Nash equilibrium and serves as a transparent baseline for studying learning dynamics. The Bayesian model captures environments in which agents face heterogeneous and privately known costs, and equilibrium strategies are functions over finite type spaces. These two settings correspond naturally to repeated interaction among fixed competitors and to environments with changing participants.

Existing work characterizes equilibria in both settings but typically relies on implicit or numerical representations. More importantly, prior analyses do not explain why independent learning algorithms should converge in these games. This paper makes three contributions.