Unified Multi-Agent Trajectory Modeling with Masked Trajectory Diffusion

Understanding movements in multi-agent scenarios is a fundamental problem in intelligent systems. Previous research assumes complete and synchronized observations. However, real-world partial observation caused by occlusions leads to inevitable model failure, which demands a unified framework for coexisting trajectory prediction, imputation, and recovery. Unlike previous attempts that handled observed and unobserved behaviors in a coupled manner, we explore a decoupled denoising diffusion modeling paradigm with a unidirectional information valve to separate the interference from uncertain behaviors. Building on this, we proposed a Unified Masked Trajectory Diffusion model (UniMTD) for arbitrary levels of missing observations. We designed a unidirectional attention as a valve unit to control the direction of information flow between the observed and masked areas, gradually refining the missing observations toward a real-world distribution. We construct it into a unidirectional MoE structure to handle varying proportions of missing observations. A Cached Diffusion model is further designed to improve generation quality while reducing computation and time overhead. Our method has achieved a great leap across human motions and vehicle traffic. UniMTD efficiently achieves 74% improvement in minADE 20 and reaches SOTA with advantages of 91%, 66%, 69%, and 58% across 4 fidelity metrics on out-of-boundary, velocity, and trajectory length. (a) UniMTD UniTraj GC-VRNN SSSD INAM Naomi MAT Transformer LSTM (b)

… Mixed Encoder Masked Observed Trajectory with arbitrary observation loss Noised Latent Space Vanilla Decoder Low-quality results Previous Methods UniMTD(Ours) Accurate Latent Space Observed Behaviors Preservation Masked Behaviors Projection Efficient Cached Diffusion Model Unidirectional Encoder High-quality results Reuse Calculation (c) Real-world Distribution