Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making

Offline decision-making via diffusion models often produces trajectories that are misaligned with system dynamics, limiting their reliability for control. We propose Model Predictive Diffuser (MPDiffuser), a compositional diffusion framework that combines a diffusion planner with a dynamics diffusion model to generate task-aligned and dynamically plausible trajectories. MPDiffuser interleaves planner and dynamics updates during sampling, progressively correcting feasibility while preserving task intent. A lightweight ranking module then selects trajectories that best satisfy task objectives. The compositional design improves sample efficiency and adaptability by enabling the dynamics model to leverage diverse and previously unseen data independently of the planner. Empirically, we demonstrate consistent improvements over prior diffusion-based methods on unconstrained (D4RL) and constrained (DSRL) benchmarks, and validate practicality through deployment on a real quadrupedal robot. Contributions. Motivated by these challenges, we propose Model Predictive Diffuser (MPDiffuser), a model-based compositional framework for offline decision making that combines three components: (i) a diffusion planner that generates diverse, task-aligned trajectories; (ii) a diffusion dynamics model that refines states to enforce consistency with system dynamics; and (iii) a ranker that selects trajectories satisfying task-specific objectives and constraints. MPDiffuser employs an alternating sampling scheme in which task-aligned proposals are repeatedly corrected by 1