CompilerDream: Learning a Compiler World Model for General Code Optimization

Effective code optimization in compilers is crucial for computer and software engineering. The success of these optimizations primarily depends on the selection and ordering of the optimization passes applied to the code. While most compilers rely on fixed pass sequences, current methods to find the optimal sequence for specific programs either employ impractically slow search algorithms or learning methods that struggle to generalize to code unseen during training. To address these challenges, we introduce CompilerDream, the first world-model-based approach for general code optimization. CompilerDream features a compiler world model with a reward smoothing technique, enabling accurate simulation of optimization processes. Built on this model, code optimization agents can then be constructed via value prediction or direct optimization sequence generation. Trained on a large-scale program dataset, these agents serve as versatile code optimizers across diverse application scenarios and source-code languages. Our extensive experiments highlight CompilerDream's strong optimization capabilities for autotuning, where it leads the CompilerGym leaderboard. More importantly, the zero-shot generalization ability of large-scale trained compiler world model and agent, excels across diverse datasets, surpassing LLVM's built-in optimizations and state-of-the-art methods in both settings of value prediction and end-to-end code optimization. 1

• Computing methodologies → Machine learning; Model development and analysis; • Software and its engineering → Compilers.