Lune

SC2025Top-tier venue

PerfDojo: Automated ML Library Generation for Heterogeneous Architectures

Andrei Ivanov, Siyuan Shen, Gioele Gottardo, Marcin Chrapek, Afif Boudaoud, Timo Schneider, Luca Benini, Torsten Hoefler

Abstract

The increasing complexity of machine learning models and the proliferation of diverse hardware architectures (CPUs, GPUs, accelerators) make achieving optimal performance a significant challenge. Heterogeneity in instruction sets, specialized kernel requirements for different data types and model features (e.g., sparsity, quantization), and architecture-specific optimizations complicate performance tuning. Manual optimization is resource-intensive, while existing automatic approaches often rely on complex hardware-specific heuristics and uninterpretable intermediate representations, hindering performance portability. We introduce PerfLLM, a novel automatic optimization methodology leveraging Large Language Models (LLMs) and Reinforcement Learning (RL). Central to this is PerfDojo, an environment framing optimization as an RL game using a human-readable, mathematically-inspired code representation that guarantees semantic validity through transformations. This allows effective optimization without prior hardware knowledge, facilitating both human analysis and RL agent training. We demonstrate PerfLLM’s ability to achieve significant performance gains across diverse CPU (x86, Arm, RISC-V) and GPU architectures.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Ansor: Generating High-Performance Tensor Programs for Deep Learning cites FlexTensor: An Automatic Schedule Exploration and Optimization Framework for Tensor Computation on Heterogeneous System
  • Tensor Program Optimization with Probabilistic Programs cites Ansor: Generating High-Performance Tensor Programs for Deep Learning
  • Tensor Program Optimization with Probabilistic Programs cites FlexTensor: An Automatic Schedule Exploration and Optimization Framework for Tensor Computation on Heterogeneous System
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsMaciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger et al.AAAI 2024 · 1,292 citations
  2. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionJay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar et al.NeurIPS 2024 · 727 citations
  3. Ansor: Generating High-Performance Tensor Programs for Deep LearningLianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu et al.OSDI 2020 · 551 citations
  4. FlexTensor: An Automatic Schedule Exploration and Optimization Framework for Tensor Computation on Heterogeneous SystemSize Zheng, Yun Liang, Shuo Wang, Renze Chen et al.ASPLOS 2020 · 171 citations
  5. Tensor Program Optimization with Probabilistic ProgramsJunru Shao, Xiyou Zhou, Siyuan Feng, Bohan Hou et al.NeurIPS 2022 · 85 citations

Related papers

  1. PerfCoder: Large Language Models for Interpretable Code Performance OptimizationJiuding Yang, Shengyao Lu, Hongxuan Liu, Shayan Shirahmad Gale Bagi et al.ACL Findings 2026
  2. STARK: Strategic Team of Agents for Refining KernelsJuncheng Dong, Yang Yang, Tao Liu, Yang Wang et al.ICLR 2026 · 26 citations
  3. TuneAgent: Agentic Operating System Kernel Tuning with Reinforcement LearningHongyu Lin, Yuchen Li, Haoran Luo, Zhenghong Lin et al.KDD 2026 · 3 citations
  4. Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement LearningHaolin Pan, Hongyu Lin, Haoran Luo, Yang Liu et al.NeurIPS 2025 · 14 citations
  5. QiMeng-Kernel: Macro-Thinking Micro-Coding Paradigm for LLM-Based High-Performance GPU Kernel GenerationXinguo Zhu, Shaohui Peng, Jiaming Guo, Yunji Chen et al.AAAI 2026 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines