UnweaveNet: Unweaving Activity Stories
Will Price, Carl Vondrick, Dima Damen
Abstract
Our lives can be seen as a complex weaving of activities; we switch from one activity to another, to maximise our achievements or in reaction to demands placed upon us. Observing a video of unscripted daily activities, we parse the video into its constituent activity threads through a process we call unweaving. To accomplish this, we introduce a video representation explicitly capturing activity threads called a thread bank, along with a neural controller capable of detecting goal changes and resuming of past activities, together forming UnweaveNet. We train and evaluate UnweaveNet on sequences from the unscripted egocentric dataset EPIC-KITCHENS. We propose and showcase the efficacy of pretraining UnweaveNet in a self-supervised manner.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
10 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers10
- EgoEnv: Human-centric environment representations from egocentric videoTushar Nagarajan, Santhosh Kumar Ramakrishnan, Ruta Desai, James Hillis et al.NeurIPS 2023 · 28 citations
- ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the BodySaif Mahmud, Vineet Parikh, Qikang Liang, Ke Li et al.UbiComp 2025 · 24 citations
- : A Visual Analytics Approach for Interactive Video ProgrammingJianben He, Xingbo Wang, Kamkwai Wong, Xijie Huang et al.IEEE VIS 2023 · 17 citations
- SpotEM: Efficient Video Search for Episodic MemorySanthosh Kumar Ramakrishnan, Ziad Al-Halah, Kristen GraumanICML 2023 · 15 citations
- Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMsZeyi Huang, Yuyang Ji, Xiaofang Wang, Nikhil Mehta et al.CVPR 2025
Builds on6
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Compositional Generalization via Neural-Symbolic Stack MachinesXinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song et al.NeurIPS 2020 · 112 citations
- Generic Event Boundary Detection: A Benchmark for Event SegmentationMike Zheng Shou, Stan Weixian Lei, Weiyao Wang, Deepti Ghadiyaram et al.ICCV 2021 · 91 citations
- Temporally-Weighted Hierarchical Clustering for Unsupervised Action SegmentationM. Saquib Sarfraz, Naila Murray, Vivek Sharma, Ali Diba et al.CVPR 2021
- A Local-to-Global Approach to Multi-Modal Movie Scene SegmentationAnyi Rao, Linning Xu, Yu Xiong, Guodong Xu et al.CVPR 2020
Related papers
- Learning State-Aware Visual Representations from Audible InteractionsHimangi Mittal, Pedro Morgado, Unnat Jain, Abhinav GuptaNeurIPS 2022 · 30 citations
- Chirality in Action: Time-Aware Video Representation Learning by Latent StraighteningPiyush Bagad, Andrew ZissermanNeurIPS 2025 · 14 citations
- Ego-Exo: Transferring Visual Representations From Third-Person to First-Person VideosYanghao Li, Tushar Nagarajan, Bo Xiong, Kristen GraumanCVPR 2021
- Shaping embodied agent behavior with activity-context priors from egocentric videoTushar Nagarajan, Kristen GraumanNeurIPS 2021 · 23 citations
- Opening the Vocabulary of Egocentric ActionsDibyadip Chatterjee, Fadime Sener, Shugao Ma, Angela YaoNeurIPS 2023 · 28 citations
