Centralized Training and Decentralized Control through the Actor-Critic Paradigm for Highly Optimized Multicores

While distributed, neural-network-based resource controllers represent the state of the art for their ability to cope with the ever-expanding decision space, such approaches suffer from several limitations, like conflicting control decisions and partial observability. These effects can significantly impair the controllers’ learning capabilities and the stability of their control policies, causing substantial performance losses. We are the first to solve this problem employing a centralized training and decentralized control regime to mitigate the aforementioned limitations. Specifically, we design a centralized neural network (critic) that evaluates the behavior of multiple decentralized neural controllers (actors) in a system-wide context. The objective of our proposed technique is to maximize the performance under a temperature constraint through dynamic voltage frequency scaling. The evaluation of our technique shows its superiority over the state of the art, yielding average (peak) performance improvements of 20% (34%), which we consider a breakthrough as the gains are measured on a real-world platform.