ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Reality
Jaewook Lee, Filippo Aleotti, Diego Mazala, Guillermo Garcia-Hernando, Sara Vicente, Oliver James Johnston, Isabel Kraus-Liang, Jakub Powierza, Donghoon Shin, Jon E. Froehlich, Gabriel J. Brostow, Jessica Van Brummelen
摘要
While augmented reality (AR) enables new ways to play, tell stories, and explore ideas rooted in the physical world, authoring personalized AR content remains difficult for non-experts, often requiring professional tools and time. Prior systems have explored AI-driven XR design but typically rely on manually defined VR environments and fixed asset libraries, limiting creative flexibility and real-world relevance. We introduce ImaginateAR, the first mobile tool for AI-assisted AR authoring to combine offline scene understanding, fast 3D asset generation, and LLMs -- enabling users to create outdoor scenes through natural language interaction. For example, saying "a dragon enjoying a campfire" (P7) prompts the system to generate and arrange relevant assets, which can then be refined manually. Our technical evaluation shows that our custom pipelines produce more accurate outdoor scene graphs and generate 3D meshes faster than prior methods. A three-part user study (N=20) revealed preferred roles for AI, how users create in freeform use, and design implications for future AR authoring tools. ImaginateAR takes a step toward empowering anyone to create AR experiences anywhere -- simply by speaking their imagination.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper7
- Storycaster: An AI System for Immersive Room-based StorytellingNaisha Agarwal, Judith Amores, Andrew D. WilsonCHI 2026 · 被引用 2 次
- NaviNote: Enabling In-situ Spatial Annotation Authoring to Support Exploration and Navigation for Blind and Low Vision PeopleRuijia Chen, Yuheng Wu, Charlie Houseago, Filipe Gaspar 等CHI 2026 · 被引用 1 次
- Roomify: Spatially-Grounded Style Transformation for Immersive Virtual EnvironmentsXueyang Wang, Qinxuan Cen, Weitao Bi, Yunxiang Ma 等CHI 2026 · 被引用 1 次
- SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB SequencesSeok-Young Kim, Dooyoung Kim, Woojin Cho, Hail Song 等IEEE VR 2026 · 被引用 1 次
- AgentHands: Generating Interactive Hand Gestures for Spatially Grounded Agent Conversations in XRZiyi Liu, David Li, Zhongyi Zhou, David Kim 等CHI 2026 · 被引用 1 次
它引用的顶会 Paper45
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao 等ICCV 2023 · 被引用 13,211 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li 等NeurIPS 2022 · 被引用 8,965 次
相关 Paper
- CoCreatAR: Enhancing Authoring of Outdoor Augmented Reality Experiences Through Asymmetric CollaborationNels Numan, Gabriel J. Brostow, Suhyun Park, Simon Julier 等CHI 2025 · 被引用 12 次
- DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote EnvironmentsZeyu Wang, Cuong Nguyen, Paul Asente, Julie DorseyCHI 2021 · 被引用 47 次
- SonifyAR: Context-Aware Sound Generation in Augmented RealityXia Su, Jon E. Froehlich, Eunyee Koh, Chang XiaoUIST 2024 · 被引用 12 次
- XRDirector: A Role-Based Collaborative Immersive Authoring SystemMichael Nebeling, Katy Lewis, Yu-Cheng Chang, Lihan Zhu 等CHI 2020 · 被引用 80 次
- VRCopilot: Authoring 3D Layouts with Generative AI Models in VRLei Zhang, Jin Pan, Jacob Gettig, Steve Oney 等UIST 2024 · 被引用 53 次
