Towards Understanding the Mechanisms of Classifier-Free Guidance

systems, yet its underlying mechanisms remain poorly understood. In this work, we begin by analyzing CFG in a simplified linear diffusion model, where we show its behavior closely resembles that observed in the nonlinear case. Our analysis reveals that linear CFG improves generation quality via three distinct components: (i) a mean-shift term that approximately steers samples in the direction of class means, (ii) a positive Contrastive Principal Components (CPC) term that amplifies class-specific features, and (iii) a negative CPC term that suppresses generic features prevalent in unconditional data. We then verify these insights in real-world, nonlinear diffusion models: over a broad range of noise levels, linear CFG resembles the behavior of its nonlinear counterpart. Although the two eventually diverge at low noise levels, we discuss how the insights from the linear analysis still shed light on the CFG's mechanism in the nonlinear regime.

Contributions. Our main contributions are as follows:

• We identify the lack of class-specificity issue of naive conditional sampling, linking it to the nondistinctiveness of class covariances. Under a linear model assumption, we show CFG overcomes this issue by amplifying class-specific features, suppressing unconditional ones and shifting the samples in the direction of class mean.

• We validate these insights derived in the linear model on real diffusion models, demonstrating that:

(i) at high to moderate noise levels, linear CFG closely matches the effects of nonlinear CFG, and (ii) at low noise levels, the insights from the linear analysis can still shed light on the mechanism of CFG in this nonlinear regime.