AraLive: Automatic Reward Adaption for Learning-based Live Video Streaming

Optimizing user Quality of Experience (QoE) for live video streaming remains a long-standing challenge. The Bitrate Control Algorithm (BCA) plays a crucial role in shaping user QoE. Recent advancements have seen RL-based algorithms overtake traditional rule-based methods, promising enhanced QoE optimization. Nevertheless, our comprehensive study reveals a pressing issue: current RL-based BCAs are limited to the fixed and formulaic reward functions, rendering them ill-equipped to adapt to dynamic network environments and varied viewer preferences. In this work, we present AraLive, an automatically adaptive reward learning method designed for seamless integration with any existing learning-based approach in live streaming contexts. To achieve this goal, we have two main designs. First, we construct a dedicated user QoE assessment dataset for live streaming, which includes thousands of videos with millisecond-level metrics. Second, we custom-design an adversarial model that skillfully aligns human feedback with actual network scenarios. We have deployed AraLive in practical video streaming systems, in comparison to a series of state-of-the-art BCAs. The experimental results demonstrate that AraLive not only elevates overall QoE but also exhibits remarkable adaptability to varied user preferences.