Improving GPU Energy Efficiency through an Application-transparent Frequency Scaling Policy with Performance Assurance
Yijia Zhang, Qiang Wang, Zhe Lin, Pengxiang Xu, Bingqiang Wang
Abstract
Power consumption is one of the top limiting factors in high-performance computing systems and data centers, and dynamic voltage and frequency scaling (DVFS) is an important mechanism to control power. Existing works using DVFS to improve GPU energy efficiency suffer from the limitation that their policies either impact performance too much or require offline application profiling or code modification, which severely limits their applicability on large clusters. To address this issue, we propose a novel GPU DVFS policy, GEEPAFS, which improves the energy efficiency of GPUs while providing performance assurance. GEEPAFS is application-transparent as it does not require any offline profiling or code modification on user applications. To achieve this, GEEPAFS models application performance online based on our quantitative analysis of a correlation between performance and GPU memory bandwidth utilization. Based on their relationship, GEEPAFS builds a fold-line frequency-performance model for applications being executed, and it applies the model to guide the setting of GPU frequency to maximize energy efficiency while ensuring the performance loss is bounded. Through experiments on NVIDIA V100 and A100 GPUs, we show that GEEPAFS is able to improve the energy efficiency by 26.7% and 20.2% on average. While achieving this improvement, the average performance loss is only 5.8%, and the worst-case performance loss is 12.5% among all 33 tested applications.
Ask Lune
Know where this paper stands in minutes, not an afternoon.
Lune can read the top-tier papers that cite it and the ones closest to it, and tells you what it claims, what came after and where it falls short, each point cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
its abstract · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LithOS: An Operating System for Efficient Machine Learning on GPUs cites DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
- Using Analytical Performance/Power Model and Fine-Grained DVFS to Enhance AI Accelerator Energy Efficiency cites DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
- Power Sloshing in Compound Servers for Large-Scale AI Inference Workloads cites DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Cited by top-tier papers6
- DynamoLLM: Designing LLM Inference Clusters for Performance and Energy EfficiencyJovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas et al.HPCA 2025 · 106 citations
- Using Analytical Performance/Power Model and Fine-Grained DVFS to Enhance AI Accelerator Energy EfficiencyZibo Wang, Yijia Zhang, Fuchun Wei, Bingqiang Wang et al.ASPLOS 2025 · 9 citations
- LithOS: An Operating System for Efficient Machine Learning on GPUsPatrick H. Coppock, Brian Zhang, Eliot H. Solomon, Vasilis Kypriotis et al.SOSP 2025 · 4 citations
- Untangling GPU Power Consumption: Job-Level Inference in Cloud Shared SettingsPierre Jacquet, Maxime Agusti, Eddy Caron, Camille Coti et al.EuroSys 2026 · 2 citations
- Power Sloshing in Compound Servers for Large-Scale AI Inference WorkloadsAlbert Cho, Jovan Stojkovic, Leonardo Piga, Abhishek Dhanotia et al.ISCA 2026 · 1 citation
Related papers
- Know Your Enemy To Save Cloud Energy: Energy-Performance Characterization of Machine Learning ServingJunyeol Yu, Jongseok Kim, Euiseong SeoHPCA 2023 · 14 citations
- Predict; Don't React for Enabling Efficient Fine-Grain DVFS in GPUsSrikant Bharadwaj, Shomit Das, Kaushik Mazumdar, Bradford M. Beckmann et al.ASPLOS 2023 · 15 citations
- PowerWeave: Unlocking Energy-Efficient ML on GPUs with OS-Level Spatial Power ManagementVasilis Kypriotis, Eric Dubberstein, Patrick H. Coppock, Eliot H. Solomon et al.ISCA 2026 · 1 citation
- GVProf: a value profiler for GPU-based clustersKeren Zhou, Yueming Hao, John M. Mellor-Crummey, Xiaozhu Meng et al.SC 2020 · 24 citations
- DrGPUM: Guiding Memory Optimization for GPU-Accelerated ApplicationsMao Lin, Keren Zhou, Pengfei SuASPLOS 2023 · 13 citations
