ExplabOff: Towards Explorative and Collaborative Task Offloading via Mutual Information-Enhanced MARL

Multi-access edge computing provides mobile devices (MDs) with both satisfactory computing resources and task latency, by offloading MDs' tasks to nearby edge servers. There is a popular trend to develop decentralized offloading (dec-offloading) approaches using multi-agent reinforcement learning (MARL), primarily based on centralized-training and decentralized-execution. However, the dec-offloading policies together could also lack exploration and collaboration since each MD is guided by the policy-critic only through offloading costs without explicitly considering the impacts of other MDs' offloading behaviors. Motivated by this, we propose Explorative and collaborative Offloading (ExplabOff) that can achieve superior dec-offloading by consciously exploiting the implicit exploration and collaboration information involved in MDs' states and actions. Specifically, we design two additional policy-learning metrics, the exploration-metric based on the maximum entropy of MDs' joint offloading actions and collaboration-metric based on one MD's belief about others' offloading behaviors. Then, we assemble these metrics into a new criterion defined as the mutual information (MI) between MDs' states and actions, and adopt it as an additive reward except for the vanilla reward during centralized-training. Furthermore, we distinguish MI between superior and inferior offloading, strengthening and weakening them discriminatively. Experiments on both simulation and real-testbed verify the effectiveness of ExplabOff over state-of-the-art dec-offloading.