LSTCB: Long-Short-Timescale Cooperative Batching for Energy-Efficient Edge Inference

Batching is a critical technique for improving energy efficiency (EE) when dealing with edge-based inference tasks. While existing batching policies achieve significant energy savings by exploiting short-timescale arrival patterns, they largely overlook long-timescale load fluctuations—leading to severe EE degradation during lightly-loaded periods, as confirmed by prototype measurements. To overcome this limitation, we propose Long-Short-Timescale Cooperative Batching (LSTCB), a novel framework that enhances EE without sacrificing latency. LSTCB dynamically deactivates servers in response to load variations while intelligently coordinating batched inference tasks. To achieve this, we first develop a measurement-based EE model for edge servers that accurately characterizes the relationship among batch size, workload, and energy consumption. At the short timescale, we devise a dynamic batching policy through Lyapunov optimization that simultaneously minimizes both energy consumption and latency under varying traffic conditions. For long-timescale optimization, we implement an intelligent server deactivation scheme that employs dynamic programming combined with bipartite graph matching to steer inter-server traffic, thereby minimizing overall system-level energy costs while maintaining service quality. Real-world prototype experiments demonstrate that LSTCB achieves 24.48%-46.12% energy savings over baseline methods while maintaining competitive latency performance.