Enabling Continuous, in-Field Introspection: The Programmable IPU Architecture

Modern silicon systems are increasingly opaque, leaving developers blind to in-field hardware behavior on real workloads. Existing introspection tools force a debilitating tradeoff: fixed-function Performance Monitoring Units (PMUs) are low-overhead but lack programmability, while trace-based solutions are high-fidelity but incur prohibitive data and performance overhead, and have an accessibility challenge as well. This leaves a critical “capability gap,” with no solution offering programmability, low overhead, and real-time analysis simultaneously. This paper introduces the Introspection Processing Unit (IPU), a new, lightweight architectural primitive designed to fill this gap. The IPU is a novel hybrid microarchitecture, pairing a programmable RISC-V core with judiciously chosen microarchitectural accelerator extensions, including a tiny eFPGA. This hybrid design is our core insight, balancing the flexibility of software with the line-rate efficiency required for non-invasive introspection. We demonstrate the IPU's power not through point solutions, but by showing its ability to perform classes of analysis previously considered intractable in the field. We show a single IPU architecture can perform: 1) Stateful Emulation: (e.g., A/B testing instruction prefetchers), 2) Software-Defined Performance Attribution (e.g., creating per-instruction cycle stacks), 3) Scalable, On-Chip Data Aggregation (e.g., fine-grained GPU utilization monitoring), and 4) Real-Time Component-Level Diagnosis (e.g., per-miss root-cause diagnosis of a prefetcher). Our complete RTL-level prototype, synthesized at 7nm, proves this new capability is practical, with an area overhead of < 1% and power consumption of $<25 m W$.