A Neural Network Approach for Efficiently Answering Most Probable Explanation Queries in Probabilistic Models
Abstract
We propose a novel neural networks based approach to efficiently answer arbitrary Most Probable Explanation (MPE) queries-a well-known NP-hard task-in large probabilistic models such as Bayesian and Markov networks, probabilistic circuits, and neural auto-regressive models. By arbitrary MPE queries, we mean that there is no predefined partition of variables into evidence and non-evidence variables. The key idea is to distill all MPE queries over a given probabilistic model into a neural network and then use the latter for answering queries, eliminating the need for time-consuming inference algorithms that operate directly on the probabilistic model. We improve upon this idea by incorporating inference-time optimization with self-supervised loss to iteratively improve the solutions and employ a teacherstudent framework that provides a better initial network, which in turn, helps reduce the number of inference-time optimization steps. The teacher network utilizes a self-supervised loss function optimized for getting the exact MPE solution, while the student network learns from the teacher's near-optimal outputs through supervised loss. We demonstrate the efficacy and scalability of our approach on various datasets and a broad class of probabilistic models, showcasing its practical effectiveness.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers2
- Learning to Condition: A Neural Heuristic for Scalable MPE InferenceBrij Malhotra, Shivvrat Arya, Tahrima Rahman, Vibhav Giridhar GogateNeurIPS 2025 · 1 citation
- Branch and Bound Search for Exact MAP Inference in Credal NetworksRadu Marinescu, Fábio G. Cozman, Denis Deratani Mauá, Debarun Bhattacharjya et al.ICLR 2026
Builds on15
- Tent: Fully Test-Time Adaptation by Entropy MinimizationDequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen et al.ICLR 2021 · 1,731 citations
- Improved Knowledge Distillation via Teacher AssistantSeyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine et al.AAAI 2020 · 1,361 citations
- Test-Time Training with Self-Supervision for Generalization under Distribution ShiftsYu Sun, Xiaolong Wang, Zhuang Liu, John Miller et al.ICML 2020 · 1,220 citations
- On the Efficacy of Knowledge DistillationJang Hyun Cho, Bharath HariharanICCV 2019 · 741 citations
- A Comprehensive Overhaul of Feature DistillationByeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park et al.ICCV 2019 · 727 citations
Related papers
- Neural Network Approximators for Marginal MAP in Probabilistic CircuitsShivvrat Arya, Tahrima Rahman, Vibhav GogateAAAI 2024 · 3 citations
- Explainability as statistical inferenceHugo Henri Joseph Senetaire, Damien Garreau, Jes Frellsen, Pierre-Alexandre MatteiICML 2023 · 4 citations
- Novel Upper Bounds for the Constrained Most Probable Explanation TaskTahrima Rahman, Sara Rouhani, Vibhav GogateNeurIPS 2021 · 2 citations
- Probabilistic Neural CircuitsPedro Zuidberg Dos MartiresAAAI 2024 · 11 citations
- Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic CircuitsXuejie Liu, Anji Liu, Guy Van den Broeck, Yitao LiangICML 2023 · 21 citations
