Low-Rank Spectral Flow Matching for Human Trajectory Prediction

To tackle the inherent multimodality challenge in human trajectory prediction, existing generative models learn the conditional distribution of future coordinates. However, spatiotemporal diversity is entangled across all timesteps, making it difficult to distinguish which trajectory dimensions require diverse prediction. Through Fisher Discriminant Ratio (FDR) analysis on Discrete Cosine Transform (DCT) coefficients, we find that behavioral diversity is non-uniformly distributed across spectral modes; specific components exhibit 20–600× higher FDR than others, indicating that diversity concentrates in a small subset of frequency components. Furthermore, trajectory energy concentrates in a few low-frequency DCT modes (over 99% retained empirically), revealing a strong low-rank structure. Motivated by this spectral sparsity in both diversity and energy, we propose Low-Rank Spectral Flow Matching (LR-SFM), which performs conditional flow matching in the truncated DCT space rather than the raw coordinate space, yielding theoretically lower generation error and higher sampling efficiency by operating in a reduced-dimensional space. To further exploit the non-uniform diversity structure, we introduce Spectral Diversity Loss with learnable per-mode margins that automatically discover which spectral modes require diverse predictions; the learned margins empirically concentrate on DC and odd modes, confirming that FDR serves as an effective proxy for identifying where diversity matters most. Experiments on the ETH–UCY, SDD, and NBA datasets show that our method consistently achieves state-of-the-art performance across multiple evaluation metrics.