ICML2026
TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions
Linli Yao, Yuancheng Wei, Yaojie Zhang, Lei Li, Xinlong Chen, Feifan Song, Ziyue Wang, Kun Ouyang, Yuanxin Liu, Lingpeng Kong, Qi Liu, Pengfei Wan, Kun Gai, Yuanxing Zhang, Xu SUN
摘要
Timestamp:00:34 -00:41 Detailed Events:From an overhead view, the white car continues to drive in circles. Inside, the curly-haired male driver is Xialuo's brotherin-law. He is a middle-aged East Asian man wearing a dark suit jacket and a dark blue shirt. He urges his brother-in-law, Xialuo, to stop showing off, explaining that he must leave immediately because it's his girlfriend's 60th birthday and he took the car without permission. Visual Background:The scene, from a bird's-eye view, shows the paved driveway of the manor and lush green trees. Acoustics Content:(1)Speaking tone:The brother-in-law's tone is impatient. (2)Background sound or music: Light, cheerful music continues to play. Dialogue Content:Driver: 'Listen, brother-in-law, that's enough. I have to hurry back. It's my girlfriend's 60th birthday today, and she has no idea I took the car out.' Camera State:The shot begins with a high-angle medium-long shot of a car. The camera then moves down and pans to the upper right, capturing a full shot of the car. Subsequently, the scene cuts to an exterior extreme close-up of the car, showing a shaky close-up of the driver through the windshield. Shot Editing Style:The editing cuts between the high-angle shot and the driver's close-up, creating a contrast between the external scene of the car driving and the internal monologue that explains the driver's motivation.