Spatio-Temporal Control Variates with ReSTIR for Real-Time Rendering

Real-time path tracing demands high visual quality under extremely tight sampling budgets, often relying on reservoir-based spatio-temporal importance resampling (ReSTIR) to maximize sample quality. However, ReSTIR typically estimates the pixel integral using a single representative sample selected via a scalar target function (e.g., luminance). This inevitably leads to color noise in scenes with complex chromatic lighting or materials. In this work, we present Reservoir-based Spatio-Temporal Control Variates (ReSTCV), a novel framework that addresses this problem by integrating Spatio-Temporal Control Variates (STCV) into ReSTIR. We revisit image-space control variates—originally an offline technique—and adapt them for real-time rendering by spatio-temporal sample reuse. This unified approach combines the benefits of both techniques, enabling us to suppress color noise while maintaining the efficiency of ReSTIR. Our method introduces minimal computational overhead and requires only minor modifications to existing ReSTIR pipelines. We demonstrate that ReSTCV produces significantly cleaner images with stable colors across a variety of dynamic scenes, marking the first practical application of spatio-temporal control variates in real-time path tracing.