Localizing, Structuring, and Rendering: Bridging 3D and 2D Vision-Language-Action Models for Robotic Manipulation

Diff Render +10.0 +10.0 +25.0 +20.0 +20.0 +20.0 Real-World Robot Manipulation Figure 1. 2D VLA models (left-top) leverage intuitive semantic perception from multi-view transformers but struggle with explicit 3D spatial reasoning. 3D VLA models (left-bottom) achieve precise spatial perception through point clouds and voxels but sacrifice visual interpretability. Our method bridges these paradigms by embedding spatial semantics into images via differentiable rendering.