Task Breakpoint Generation using Origin-Centric Graph in Virtual Reality Recordings for Adaptive Playback
Selin Choi, Dooyoung Kim, Taewook Ha, Seonji Kim, Woontack Woo
Abstract
We propose a method for generating task breakpoints based on an Origin-Centric Graph (OCG) to segment goal-oriented activity recordings into task units for adaptive playback in Virtual Reality (VR) environments. With the development of Augmented Reality (AR)/VR head-mounted displays (HMDs), research on adaptive tutorials and authoring tools has become active, but existing task segmentation methods mainly rely on manual annotation or are restricted to 2D video which limits their applicability to 3D VR contexts. In our approach, assembly scenarios with clearly defined task boundaries are recorded using a structured spatio-temporal scene graph (STSG), and the OCG is employed to track changes in the central object and the formation of new groups, thereby generating task breakpoints automatically. A user study collected user-perceived task breakpoints to establish ground truth (GT), and comparison with the algorithm-detected breakpoints demonstrated high agreement and confirmed accuracy in supporting adaptive playback. The proposed task segmentation method provides a foundation for dynamically adjusting VR playback according to user proficiency and progress, with potential for extension into automatic timeline segmentation systems for diverse VR recordings.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 7f0c00e4-b08e-4e13-b9b6-8b17cca8209dBuilds on13
- SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 4,104 citations
- ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User StudiesSebastian Hubenschmid, Jonathan Wieland, Daniel Immanuel Fink, Andrea Batch et al.CHI 2022 · 99 citations
- AdapTutAR: An Adaptive Tutoring System for Machine Tasks in Augmented RealityGaoping Huang, Xun Qian, Tianyi Wang, Fagun Patel et al.CHI 2021 · 93 citations
- Generic Event Boundary Detection: A Benchmark for Event SegmentationMike Zheng Shou, Stan Weixian Lei, Weiyao Wang, Deepti Ghadiyaram et al.ICCV 2021 · 91 citations
- Compositional Video Synthesis with Action GraphsAmir Bar, Roei Herzig, Xiaolong Wang, Anna Rohrbach et al.ICML 2021 · 48 citations
Related papers
- Video-Annotated Augmented Reality Assembly TutorialsMasahiro Yamaguchi, Shohei Mori, Peter Mohr, Markus Tatzgern et al.UIST 2020 · 37 citations
- Generating Activity Snippets by Learning Human-Scene InteractionsChangyang Li, Lap-Fai YuSIGGRAPH 2023 · 10 citations
- HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video UnderstandingTrong-Thuan Nguyen, Pha A. Nguyen, Khoa LuuCVPR 2024 · 5 citations
- Using Virtual Replicas to Improve Mixed Reality Remote CollaborationHuayuan Tian, Gun A. Lee, Huidong Bai, Mark BillinghurstIEEE VR 2023 · 57 citations
- Progress-Aware Online Action Segmentation for Egocentric Procedural Task VideosYuhan Shen, Ehsan ElhamifarCVPR 2024 · 14 citations
