EigenCache: Rethinking Diffusion Acceleration as Covariance-Optimal Forecasting and Submodular Information Allocation

Accelerating diffusion models via feature caching has progressed from static feature reuse to polynomial extrapolation, yet current "cache-thenforecast" strategies still rely on hand-crafted approximation families (e.g., Taylor or Hermite bases) that can misalign with the non-stationary, layer-specific dynamics of generative features. This paper introduces EigenCache, a trainingfree framework that re-frames diffusion acceleration as covariance-adaptive feature forecasting and uncertainty-aware temporal design. Eigen-Cache models cached feature trajectories as timeindexed stochastic processes and estimates layerwise temporal kernels from a small calibration set. Under the resulting scalar temporalkernel approximation, the Gaussian-process posterior mean, i.e., Kriging, is risk-optimal within the scalar-weighted linear predictor class; under joint Gaussianity, it further coincides with the MMSE estimator. This formulation generalizes fixed-basis forecasting from a covarianceadaptive perspective while providing a closedform posterior-variance proxy for prediction uncertainty. Leveraging this proxy, EigenCache selects computation anchors by maximizing a logdeterminant information-gain objective over denoising timesteps, a monotone submodular objective with a near-optimal greedy solution. Across image, video, transformer, U-Net, and LoRAadapted diffusion models, EigenCache achieves a strong speed-fidelity Pareto frontier and provides a principled mechanism for robust compute allocation. Our code is available here.