NetTrack: Tracking Highly Dynamic Objects with a Net

Open-world prompt: "Osprey (Unknown) + fish (Known)" Deformation! Deformation! Referring prompt : "A large bird with brown head, wings, and white tail is predating" Fast motion! Deformation! b a Track with a Box (traditional) Fine-grained object cues Track with a Net (ours) Coarse-grained object cues Internal relationships Failure Robust Coarse-grained Fine-grained Figure 1. a The visualization of the proposed NetTrack is similar to a Net. Object dynamicity distorts the internal relationships of the object, presenting challenges for traditional coarse-grained tracking methods that rely solely on bounding boxes. While NetTrack introduces fine-grained Nets that are robust to dynamicity. b Qualitative results of NetTrack tracking highly dynamic objects under openworld tracking and referring expression comprehension settings. Dynamicity like deformation and fast motion results in drastic changes in the coarse-grained representation, while the fine-grained Nets can contract robustly. The dashed boxes represent the object position from the previous time step. c We propose a challenging benchmark named BFT, dedicated to evaluating highly dynamic object tracking with abundant scenarios shown in the external circular and diverse species shown in the central word cloud.