The City as a Personal Assistant: Turning Urban Landmarks into Conversational Agents for Serving Hyper Local Information

Conversational agents are increasingly becoming digital partners in our everyday computational experiences. Although rich and fresh in content, they are oblivious to users' locality beyond geospatial weather and traffic conditions. We introduce Lingo, a hyper-local conversational agent embedded deeply into the urban infrastructure that provides rich, purposeful, detailed, and in some cases, playful information relevant to a neighbourhood. Drawing lessons from a mixed-method contextual study (online survey, 𝑛 = 1992 and semi-structured interviews, 𝑛 = 21), we identify requirements for such a hyper-local conversational agent and a sample set of questions serving urban neighbourhoods of Belgium. Our agent design is manifested into a two-part system. First, a multi-modal reasoning engine serves as a hyper-local information source using automated machine-learning models operating on camera, microphone, and environmental sensor data. Second, a smart conversational speaker and a smartphone application serve as hyper-local information access points. Finally, we introduce a covert communication mechanism over Wi-Fi management frames that bridges the two parts of our Lingo system and enables the privacy-preserving proxemic interactions. We describe the design, implementation, and technical assessment of Lingo together with usability (𝑛 = 20) and real-world deployment (𝑛 = 5) studies. We reflect on information quality, accessibility benefits, and interaction dynamics and demonstrate the efficacy of Lingo in offering hyper-local information at the finest granularity in urban neighbourhoods while reducing access time up to a factor of 25.

CCS Concepts: • Human-centered computing → Sound-based input / output; Ubiquitous and mobile computing systems and tools; Ubiquitous computing; Mobile computing.