MASS: Motion-Aware Spatial–temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models

Vision Language Models (VLMs) perform well on standard video tasks but struggle with physics-related reasoning that involves motion dynamics and spatial interactions. We present a novel approach to address this gap by translating physical-world context cues into interpretable representations aligned with VLMs' perception, comprehension, and reasoning. We present MASS, a model-agnostic approach that injects spatial-temporal signals into the VLM language space via depth-based 3D encoding and visual grounding, coupled with a motion tracker for object dynamics. We also contribute a comprehensive benchmark, MASS-Bench, consisting of 4, 350 real-world and AIGC videos and 8, 361 freeform video question-answering pairs focused on physicsrelated comprehension tasks, with detailed annotations including visual detections and grounding over sub-segments, as well as full-sequence 3D motion tracking of entities. To strengthen cross-modal alignment and reasoning, we apply reinforcement fine-tuning to MASS. Experiments and ablations show that our refined VLMs outperform comparable and baselines of larger size, and prior state-of-the-art (SoTA) models, achieving performance comparable to closesource SoTA VLMs (2% difference) like Gemini-2.5-Flash on physics reasoning and comprehension.