A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation
Wei Chen, Jiacheng Li, Shigui Li, Zhiqi Lin, Junmei Yang, John Paisley, Delu Zeng
Abstract
Score-based methods are powerful across machine learning, but they face a paradox: theoretically path-independent, yet practically path-dependent. We resolve this by proving that practical training objectives differ from the ideal, ground-truth objective by a crucial, overlooked term: the path variance of the score function. We propose the MVP (Mimum Variance Path) Principle to minimize this path variance. Our key contribution is deriving a closed-form expression for the variance, making optimization tractable. By parameterizing the path with a flexible Kumaraswamy Mixture Model, our method learns data-adaptive, low-variance paths without heuristic manual selection. This principled optimization of the complete objective yields more accurate and stable estimators, establishing new state-of-the-art results on challenging benchmarks and providing a general framework for optimizing score-based interpolation. Our code can be found in https://github.com/Hoemr/OpenDRE.git.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers2
- Rényi Diffusion ModelsYirong Shen, Lu GAN, Cong LingICML 2026 · 4 citations
- Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in GenomicsEgor Antipov, Alessandro Palma, Lorenzo Consoli, Stephan Günnemann et al.ICML 2026
Builds on29
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Improved Denoising Diffusion Probabilistic ModelsAlexander Quinn Nichol, Prafulla DhariwalICML 2021 · 5,234 citations
- Score-Based Generative Modeling through Stochastic Differential EquationsYang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar et al.ICLR 2021 · 1,270 citations
- Maximum Likelihood Training of Score-Based Diffusion ModelsYang Song, Conor Durkan, Iain Murray, Stefano ErmonNeurIPS 2021 · 958 citations
- Guiding a Diffusion Model with a Bad Version of ItselfTero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen et al.NeurIPS 2024 · 338 citations
Related papers
- Traversing Distortion-Perception Tradeoff using a Single Score-Based Generative ModelYuhan Wang, Suzhi Bi, Ying-Jun Angela Zhang, Xiaojun YuanCVPR 2025
- SFESS: Score Function Estimators for k-Subset SamplingKlas Wijk, Ricardo Vinuesa, Hossein AzizpourICLR 2025
- Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable ModelsFan Bao, Kun Xu, Chongxuan Li, Lanqing Hong et al.ICML 2021 · 10 citations
- Density Ratio Estimation with Conditional Probability PathsHanlin Yu, Arto Klami, Aapo Hyvärinen, Anna Korba et al.ICML 2025
- Generalization through variance: how noise shapes inductive biases in diffusion modelsJohn J. VastolaICLR 2025
