Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation

Figure 1: In contrast to previous methods that (a) conduct end-to-end naive behavior cloning from large-scale robotic data or (b) use decoupled visual prediction and inverse dynamics models to set goals and guide actions, we present end-to-end Predictive Inverse Dynamics Models (PIDM) that closes the loop between vision and action. Seer, the model we built, surpasses previous states of the art and demonstrates consistent improvements over the ablated version without pre-training.