Conducting Conditional Diffusion by Estimating the Mean Vector of von Mises-Fisher Distribution

Recent diffusion model advancements aim to handle conditional generative tasks without extra training. Existing training-free methods add a correction term at each denoising step, but they often face computational instability and lack controllability, especially with limited samples and large noise. We propose a new approach using the von Mises-Fisher (vMF) distribution to model the denoised result, turning the conditional generation task into an estimation problem for vMF parameters. We formulate the conditional diffusion model as a mean vector estimation problem for the Gaussian distribution, noting that this can be seen as an estimation problem from noisy observations. When the sampling number is small, the estimation is unstable. To address this, we optimize the mean vector of the vMF distribution by minimizing the KL divergence between the prior and posterior distributions. This approach not only addresses the computational instability but also improves the controllability and quality of the generated results. Once these parameters are determined, the denoised result can be sampled directly from the vMF distribution. Estimating the parameters requires minimal additional code and incurs negligible computational overhead while significantly improving performance. Extensive experiments across various conditional generation tasks, including depth maps, edge detection, segmentation, and style guidance, demonstrate the superiority and versatility of our method. Our approach consistently outperforms existing training-free methods and even surpasses some training-required methods in terms of visual quality and controllability.