CoopRide: Cooperate All Grids in City-Scale Ride-Hailing Dispatching with Multi-Agent Reinforcement Learning

Ride-hailing services offer convenient travel options in urban transportation. To improve passengers' experience and platforms' revenue, plentiful studies use multi-agent reinforcement learning (MARL) for efficient order dispatching, controlling each grid with one agent to balance the supply-demand (drivers-orders) distribution. However, despite the critical role of cooperation among grids for efficient dispatching strategies, existing works neglect it or limit it within neighboring grids. There exist three key challenges in scaling the cooperation to the whole city: (1) cooperative strategies cause complex interactions among grids, making the grids' states coupled and complicating the information extraction from the states for decision-making; (2) cooperation among grids requires both within- and cross-grid dispatching, where the priorities of these two types of actions are difficult to balance; (3) the value of cooperation is not only heterogeneous over different pairs of grids, but also varies temporally, adding difficulty to dynamically determine the intensities of cooperation for each pair of grids and obtain the global cooperation rewards. In this paper, we propose the CoopRide framework to solve the above challenges. We model the interactions among agents with graphs and utilize graph neural network (GNN) for efficient information extraction. We uniformly encode both within- and cross-grid dispatching, enabling flexible choice of both types of actions in the embedding space. We also design to automatically learn the cooperation intensities among grids, thereby obtaining the cooperative rewards to drive the learning of global cooperation actions. We conduct experiments in three real-world datasets with millions of orders, and extensive results demonstrate the superior performance of CoopRide, outperforming the state-of-the-art baselines by up to 12.4%. Our source codes are available at https://github.com/tsinghua-fib-lab/CoopRide.