SceneScout: Towards AI-Driven Access to Street Level Imagery for Blind Users
Gaurav Jain, Leah Findlater, Cole Gleason
摘要
People who are blind or have low-vision (BLV) may hesitate to travel independently in unfamiliar environments due to uncertainty about the physical landscape. While most tools focus on in-situ navigation assistance, those supporting pre-travel assistance typically provide information about only landmarks and turn-by-turn instructions, lacking detailed visual context. Street level imagery, which contains rich visual information and has the potential to reveal numerous environmental details, remains inaccessible to BLV people. In this work, we present SceneScout, a multimodal large language model (MLLM)-driven prototype that enables accessible interactions with street level imagery. SceneScout supports two modes: (1) Route Preview, enabling users to familiarize themselves with visual details along a route, and (2) Virtual Exploration, enabling free, user-driven movement within street level imagery. Our user study (N = 10) demonstrates that SceneScout helps BLV users uncover visual information otherwise unavailable through existing means. An initial analysis of AI-generated descriptions suggests that the majority are accurate and describe stable visual elements even in older imagery, though occasional subtle and plausible errors make them difficult to verify without sight. We discuss future opportunities and challenges of street level imagery-based navigation experiences.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AIJon E. Froehlich, Alexander J. Fiannaca, Nimer Jaber, Victor Tsaran 等UIST 2025 · 被引用 5 次
- How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision PeopleRicardo E. Gonzalez Penuela, Crescentia Jung, Sharon Y. Lin, Ruiying Hu 等CHI 2026 · 被引用 1 次
- Understanding the Use of a Large Language Model-Powered Guide to Make Virtual Reality Accessible for Blind and Low Vision PeopleJazmin Collins, Sharon Y. Lin, Tianqi Liu, Andrea Stevenson Won 等CHI 2026 · 被引用 3 次
- Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision PeopleRicardo E. Gonzalez Penuela, Jazmin Collins, Cynthia L. Bennett, Shiri AzenkotCHI 2024 · 被引用 44 次
- Towards LLM-powered Assistive Drone for Blind and Low Vision UsersYize Wei, Ibnu Taimiyyah Bin Adam, Hanjun Wu, Moritz Messerschmidt 等CHI 2026 · 被引用 1 次
