SceneScout: Towards AI-Driven Access to Street Level Imagery for Blind Users
Gaurav Jain, Leah Findlater, Cole Gleason
Abstract
People who are blind or have low-vision (BLV) may hesitate to travel independently in unfamiliar environments due to uncertainty about the physical landscape. While most tools focus on in-situ navigation assistance, those supporting pre-travel assistance typically provide information about only landmarks and turn-by-turn instructions, lacking detailed visual context. Street level imagery, which contains rich visual information and has the potential to reveal numerous environmental details, remains inaccessible to BLV people. In this work, we present SceneScout, a multimodal large language model (MLLM)-driven prototype that enables accessible interactions with street level imagery. SceneScout supports two modes: (1) Route Preview, enabling users to familiarize themselves with visual details along a route, and (2) Virtual Exploration, enabling free, user-driven movement within street level imagery. Our user study (N = 10) demonstrates that SceneScout helps BLV users uncover visual information otherwise unavailable through existing means. An initial analysis of AI-generated descriptions suggests that the majority are accurate and describe stable visual elements even in older imagery, though occasional subtle and plausible errors make them difficult to verify without sight. We discuss future opportunities and challenges of street level imagery-based navigation experiences.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get bb649f57-03a4-4dd0-9c24-354b091c7be2Related papers
- StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AIJon E. Froehlich, Alexander J. Fiannaca, Nimer Jaber, Victor Tsaran et al.UIST 2025 · 5 citations
- How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision PeopleRicardo E. Gonzalez Penuela, Crescentia Jung, Sharon Y. Lin, Ruiying Hu et al.CHI 2026 · 1 citation
- Understanding the Use of a Large Language Model-Powered Guide to Make Virtual Reality Accessible for Blind and Low Vision PeopleJazmin Collins, Sharon Y. Lin, Tianqi Liu, Andrea Stevenson Won et al.CHI 2026 · 3 citations
- Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision PeopleRicardo E. Gonzalez Penuela, Jazmin Collins, Cynthia L. Bennett, Shiri AzenkotCHI 2024 · 44 citations
- Towards LLM-powered Assistive Drone for Blind and Low Vision UsersYize Wei, Ibnu Taimiyyah Bin Adam, Hanjun Wu, Moritz Messerschmidt et al.CHI 2026 · 1 citation
