Lune

NeurIPS2024Top-tier venue

A Neural Network Approach for Efficiently Answering Most Probable Explanation Queries in Probabilistic Models

Shivvrat Arya, Tahrima Rahman, Vibhav Gogate

Abstract

We propose a novel neural networks based approach to efficiently answer arbitrary Most Probable Explanation (MPE) queries-a well-known NP-hard task-in large probabilistic models such as Bayesian and Markov networks, probabilistic circuits, and neural auto-regressive models. By arbitrary MPE queries, we mean that there is no predefined partition of variables into evidence and non-evidence variables. The key idea is to distill all MPE queries over a given probabilistic model into a neural network and then use the latter for answering queries, eliminating the need for time-consuming inference algorithms that operate directly on the probabilistic model. We improve upon this idea by incorporating inference-time optimization with self-supervised loss to iteratively improve the solutions and employ a teacherstudent framework that provides a better initial network, which in turn, helps reduce the number of inference-time optimization steps. The teacher network utilizes a self-supervised loss function optimized for getting the exact MPE solution, while the student network learns from the teacher's near-optimal outputs through supervised loss. We demonstrate the efficacy and scalability of our approach on various datasets and a broad class of probabilistic models, showcasing its practical effectiveness.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Learning to Condition: A Neural Heuristic for Scalable MPE InferenceBrij Malhotra, Shivvrat Arya, Tahrima Rahman, Vibhav Giridhar GogateNeurIPS 2025 · 1 citation
  2. Branch and Bound Search for Exact MAP Inference in Credal NetworksRadu Marinescu, Fábio G. Cozman, Denis Deratani Mauá, Debarun Bhattacharjya et al.ICLR 2026

Builds on15

  1. Tent: Fully Test-Time Adaptation by Entropy MinimizationDequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen et al.ICLR 2021 · 1,731 citations
  2. Improved Knowledge Distillation via Teacher AssistantSeyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine et al.AAAI 2020 · 1,361 citations
  3. Test-Time Training with Self-Supervision for Generalization under Distribution ShiftsYu Sun, Xiaolong Wang, Zhuang Liu, John Miller et al.ICML 2020 · 1,220 citations
  4. On the Efficacy of Knowledge DistillationJang Hyun Cho, Bharath HariharanICCV 2019 · 741 citations
  5. A Comprehensive Overhaul of Feature DistillationByeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park et al.ICCV 2019 · 727 citations

Related papers

  1. Neural Network Approximators for Marginal MAP in Probabilistic CircuitsShivvrat Arya, Tahrima Rahman, Vibhav GogateAAAI 2024 · 3 citations
  2. Explainability as statistical inferenceHugo Henri Joseph Senetaire, Damien Garreau, Jes Frellsen, Pierre-Alexandre MatteiICML 2023 · 4 citations
  3. Novel Upper Bounds for the Constrained Most Probable Explanation TaskTahrima Rahman, Sara Rouhani, Vibhav GogateNeurIPS 2021 · 2 citations
  4. Probabilistic Neural CircuitsPedro Zuidberg Dos MartiresAAAI 2024 · 11 citations
  5. Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic CircuitsXuejie Liu, Anji Liu, Guy Van den Broeck, Yitao LiangICML 2023 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines