Lune

OOPSLA2025Top-tier venue

Divining Profiler Accuracy: An Approach to Approximate Profiler Accuracy through Machine Code-Level Slowdown

Humphrey Burchell, Stefan Marr

Abstract

Optimizing performance on top of modern runtime systems with just-in-time (JIT) compilation is a challenge for a wide range of applications from browser-based applications on mobile devices to large-scale server applications. Developers often rely on sampling-based profilers to understand where their code spends its time. Unfortunately, sampling of JIT-compiled programs can give inaccurate and sometimes unreliable results. To assess accuracy of such profilers, we would ideally want to compare their results to a known ground truth. With the complexity of today's software and hardware stacks, such ground truth is unfortunately not available. Instead, we propose a novel technique to approximate a ground truth by accurately slowing down a Java program at the machine-code level, preserving its optimization and compilation decisions as well as its execution behavior on modern CPUs. Our experiments demonstrate that we can slow down benchmarks by a specific amount, which is a challenge because of the optimizations in modern CPUs, and we verified with hardware profiling that on a basic-block level, the slowdown is accurate for blocks that dominate the execution. With the benchmarks slowed down to specific speeds, we confirmed that Async-profiler, JFR, JProfiler, and YourKit maintain original performance behavior and assign the same percentage of run time to methods. Additionally, we identify cases of inaccuracy caused by missing debug information, which prevents the correct identification of the relevant source code. Finally, we tested the accuracy of sampling profilers by approximating the ground truth by the slowing down of specific basic blocks and found large differences in accuracy between the profilers. We believe, our slowdown-based approach is the first practical methodology to assess the accuracy of sampling profilers for JIT-compiling systems and will enable further work to improve the accuracy of profilers.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on3

  1. Rethinking Java Performance AnalysisStephen M. Blackburn, Zixian Cai, Rui Chen, Xi Yang et al.ASPLOS 2025 · 19 citations
  2. TIP: Time-Proportional Instruction ProfilingBjörn Gottschall, Lieven Eeckhout, Magnus JahreMICRO 2021 · 16 citations
  3. Where Did My Variable Go? Poking Holes in Incomplete Debug InformationCristian Assaiante, Daniele Cono D'Elia, Giuseppe Antonio Di Luna, Leonardo QuerzoniASPLOS 2023 · 14 citations

Related papers

  1. Understanding and Finding JIT Compiler Performance BugsZijian Yi, Cheng Ding, August Shi, Milos GligoricOOPSLA 2026
  2. OJXPERF: Featherlight Object Replica Detection for Java ProgramsBolun Li, Hao Xu, Qidong Zhao, Pengfei Su et al.ICSE 2022 · 11 citations
  3. Uncovering Hidden Memory Costs for Garbage CollectionSudhanshu Agarwal, Saugata GhoseOOPSLA 2026
  4. JavART: A Lightweight Rule-Based JIT Compiler using Translation Rules Extracted from a Learning ApproachHanzhang Wang, Wei Peng, Wenwen Wang, Yunping Lu et al.OOPSLA 2025
  5. AI-driven Java Performance Testing: Balancing Result Quality with Testing TimeLuca Traini, Federico Di Menna, Vittorio CortellessaASE 2024 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines