Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effects

Input Videos Output Contact points + Forces Simulated Result Applying Forces Figure 1: We study the problem of physical understanding of human-object interactions by inferring object motion, points of contacts and forces from RGB videos. We use physics simulation to see how these interactions change the dynamics of the object and try to imitate the motion observed in the video.