Generative Point Tracking and Forecasting

Unobserved Future Figure 1. Unifying point tracking and forecasting by generative modeling. We train a model that can perform both point tracking and forecasting, by posing both problems as video-conditioned point generation. Here, we condition the model on an incomplete video sequence. When video frames are available, the model performs point tracking (left). For the unobserved frames at the end of the video, the model naturally performs trajectory forecasting (right). Background points are colored gray to highlight the motion of the main subject.