Learning Asynchronous and Sparse Human-Object Interaction in Videos

Figure 1: An example of human-object interaction activity, where a person takes some medicine and interacts with two objects. Human (circles) and object (triangles) entities have independent lives throughout the video (upper three rows). Although videos are captured in regular timing (lower rows), the dynamics of human activities and object affordance evolve sparsely and asynchronous with each other (colored segments). They also affect each other (blue curved arrows). These characteristics of human-object interactions are the main modeling goals of this work.