Learning Interactions and Relationships Between Movie Characters

I think I'm in love with you. -You mean, as friends? -No. I mean, for real. -Someday, you'll be back here, honey… on your terms. Mac is Juno's father Juno suggests to Mark Vijay runs with Paulie Juno confesses to Paulie Mac consoles Juno (Mac, Juno) (Juno, Paulie) (Juno, Mark) (Vijay, Paulie) Juno and Paulie are lovers Juno and Mark are strangers Vijay and Paulie are classmates Figure 1: The goal of this work is to jointly predict interactions and relationships between all characters in movies. Some interactions are based on dialog (e.g. suggests, confesses), some are primarily visual (e.g. runs with), and others are based on a fusion of both modalities (e.g. consoles). The colored rows at the bottom highlight when a pair of characters appear in the movie timeline. Their (directed) relationships are presented at the right. Example clips obtained from the movie Juno, 2007.