Beyond Instructions: A Taxonomy of Information Types in How-to Videos
Saelyne Yang, Sangkyung Kwak, Juhoon Lee, Juho Kim
Abstract
How-to videos are rich in information-they not only give instructions but also provide justifcations or descriptions. People seek diferent information to meet their needs, and identifying diferent types of information present in the video can improve access to the desired knowledge. Thus, we present a taxonomy of information types in how-to videos. Through an iterative open coding of 4k sentences in 48 videos, 21 information types under 8 categories emerged. The taxonomy represents diverse information types that instructors provide beyond instructions. We frst show how our taxonomy can serve as an analytical framework for video navigation systems. Then, we demonstrate through a user study (n=9) how type-based navigation helps participants locate the information they needed. Finally, we discuss how the taxonomy enables a wide range of video-related tasks, such as video authoring, viewing, and analysis. To allow researchers to build upon our taxonomy, we release a dataset of 120 videos containing 9.9k sentences labeled using the taxonomy.
• Human-centered computing → Human computer interaction (HCI).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers8
- Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming EducationHyoungwook Jin, Seonghee Lee, Hyungyu Shin, Juho KimCHI 2024 · 94 citations
- Critical Heritage Studies as a Lens to Understand Short Video Sharing of Intangible Cultural Heritage on DouyinHuanchen Wang, Minzhu Zhao, Wanyang Hu, Yuxin Ma et al.CHI 2024 · 17 citations
- AQuA: Automated Question-Answering in Software Tutorial Videos with Visual AnchorsSaelyne Yang, Jo Vermeulen, George W. Fitzmaurice, Justin MatejkaCHI 2024 · 15 citations
- NotePlayer: Engaging Computational Notebooks for Dynamic Presentation of Analytical ProcessesYang Ouyang, Leixian Shen, Yun Wang, Quan LiUIST 2024 · 11 citations
- Vid2Coach: Transforming How-To Videos into Task AssistantsMina Huh, Zihui Xue, Ujjaini Das, Kumar Ashutosh et al.UIST 2025 · 9 citations
Builds on13
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsAntoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi et al.ICCV 2019 · 1,437 citations
- What Makes Videos Accessible to Blind and Visually Impaired People?Xingyu Liu, Patrick Carrington, Xiang 'Anthony' Chen, Amy PavelCHI 2021 · 78 citations
- Automatic Generation of Two-Level Hierarchical Tutorials from Instructional Makeup VideosAnh Truong, Peggy Chi, David Salesin, Irfan Essa et al.CHI 2021 · 57 citations
- Say It All: Feedback for Improving Non-Visual Presentation AccessibilityYi-Hao Peng, JiWoong Jang, Jeffrey P. Bigham, Amy PavelCHI 2021 · 46 citations
Related papers
- YTCommentQA: Video Question Answerability in Instructional VideosSaelyne Yang, Sunghyun Park, Yunseok Jang, Moontae LeeAAAI 2024 · 6 citations
- On Pause: How Online Instructional Videos are Used to Achieve Practical TasksSylvaine Tuncer, Barry A. T. Brown, Oskar LindwallCHI 2020 · 30 citations
- Learning To Recognize Procedural Activities with Distant SupervisionXudong Lin, Fabio Petroni, Gedas Bertasius, Marcus Rohrbach et al.CVPR 2022 · 55 citations
- Modeling Health Video Consumption Behaviors on Social Media: Activities, Challenges, and CharacteristicsJiaying Liu, Yan ZhangCSCW 2024 · 20 citations
- Where Are You Looking?: A Large-Scale Dataset of Head and Gaze Behavior for 360-Degree Videos and a Pilot StudyYili Jin, Junhua Liu, Fangxin Wang, Shuguang CuiACM MM 2022 · 37 citations
