Techie: Tackling Video Prefetching at Edge Networks as POMDP Via an Intrinsically Motivated RL Agent

Video prefetching techniques play a critical role in hiding the I/O latency of video delivery from cloud networks to access networks by optimizing edge networks. Content Delivery Networks rely on edge computing to improve Quality of Experience at the client-side, and improve resource utilization at edge and cloud networks simultaneously by utilizing prefetching optimization techniques. Recently, we have witnessed a trend in shifting intelligence to edge networks by building video prefetchers that utilize deep reinforcement learning to make online prefetching decisions. Unfortunately, these methods often lack generalization to different workloads when dealing with long sequences, rendering them incapable of adapting to distribution shifts and various changes in users' requests which represent a non i.i.d distribution. In this work, we tackle video prefetching at edge networks as a Partially Observable Markov Decision Process, and propose Techie, an intrinsically motivated policy-gradient reinforcement learning agent that differentiates intrinsic rewards from extrinsic rewards based on their availability to edge networks. Techie is adaptive to unseen user requests by prefetching aggressively to handle randomization in workloads, achieving 52.27% of prefetching accuracy and 34.34% of prefetching coverage. Our results show that Techie improves prefetching accuracy and coverage by at least 16.12% and 6.57%, respectively, compared to baseline approaches that utilize deep learning, deep reinforcement learning, or video popularity to build prefetching algorithms. Consequently, Techie minimizes end-to-end latency by at least 7.5% and reduces cache pollution with an improvement of at least 13.1% on unseen user requests compared to baselines.