Concerto: Client-server Orchestration for Real-Time Video Analytics

The delay to obtain analysis results is an important metric in video analytics. Previous work has focused on reducing frame transmission and inference delay to optimize total delay. However, network fluctuations can cause frames to arrive at the backend simultaneously, leading to backend queuing delays. To address this issue, we propose Concerto, a joint front-and backend video analytics pipeline that optimizes both network transmission and backend queuing delays. The backend controls the frame queue, accelerating or skipping inference as needed to mitigate backend queuing delay. The frontend considers both delays when configuring frames to send, resulting in better total delay. Experiments show that Concerto significantly reduces backend queuing delay with minimal loss of accuracy.