Lune

CHI2025Top-tier venue

CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence

Jingyu Shi, Rahul Jain, Seunggeun Chi, Hyungjun Doh, Hyung-Gun Chi, Alexander J. Quinn, Karthik Ramani

Abstract

Context-aware AR instruction enables adaptive and in-situ learning experiences. However, hardware limitations and expertise requirements constrain the creation of such instructions. With recent developments in Generative Artificial Intelligence (Gen-AI), current research tries to tackle these constraints by deploying AI-generated content (AIGC) in AR applications. However, our preliminary study with six AR practitioners revealed that the current AIGC lacks contextual information to adapt to varying application scenarios and is therefore limited in authoring. To utilize the strong generative power of GenAI to ease the authoring of AR instruction while capturing the context, we developed CARING-AI, an AR system to author context-aware humanoid-avatar-based instructions with GenAI. By navigating in the environment, users naturally provide contextual information to generate humanoid-avatar animation as AR instructions that blend in the context spatially and temporally. We showcased three application scenarios of CARING-AI: Asynchronous Instructions, Remote Instructions, and Ad Hoc Instructions based on a design space of AIGC in AR Instructions. With two user studies (N=12), we assessed the system usability of CARING-AI and demonstrated the easiness and effectiveness of authoring with Gen-AI.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Segment Anything cites Language Models are Few-Shot Learners
  • Segment Anything cites Learning Transferable Visual Models From Natural Language Supervision
  • Diffusion Models Beat GANs on Image Synthesis cites Language Models are Few-Shot Learners
  • Diffusion Models Beat GANs on Image Synthesis cites Learning Transferable Visual Models From Natural Language Supervision
  • Diffusion Models Beat GANs on Image Synthesis cites Denoising Diffusion Probabilistic Models
  • Occlusion-Aware Temporally Consistent Amodal Completion for 3D Human-Object Interaction Reconstruction cites Learning Transferable Visual Models From Natural Language Supervision
  • Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision ModelsAda Yi Zhao, Aditya Gunturu, Ellen Yi-Luen Do, Ryo SuzukiUIST 2025 · 12 citations
  2. I Can't Join, but I Will Send My Agent: Stand-in Enhanced Asynchronous Meetings (SEAM)Zhongyi Bai, Jens Emil Sloth Grønbæk, Andrew Irlitti, Jarrod Knibbe et al.UbiComp 2026 · 2 citations
  3. Occlusion-Aware Temporally Consistent Amodal Completion for 3D Human-Object Interaction ReconstructionHyungjun Doh, Dong In Lee, Seunggeun Chi, Pin-Hao Huang et al.ACM MM 2025 · 1 citation

Builds on38

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  3. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  4. Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
  5. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations

Related papers

  1. MineVRA: Exploring the Role of Generative AI-Driven Content Development in XR Environments through a Context-Aware ApproachLorenzo Stacchio, Emanuele Balloni, Emanuele Frontoni, Marina Paolanti et al.IEEE VR 2025 · 11 citations
  2. "I'd Never Actually Realized How Big An Impact It Had Until Now": Perspectives of University Students with Disabilities on Generative Artificial IntelligenceAlex Atcheson, Omar Khan, Brian Siemann, Anika Jain et al.CHI 2025 · 22 citations
  3. VRCopilot: Authoring 3D Layouts with Generative AI Models in VRLei Zhang, Jin Pan, Jacob Gettig, Steve Oney et al.UIST 2024 · 53 citations
  4. CAPturAR: An Augmented Reality Tool for Authoring Human-Involved Context-Aware ApplicationsTianyi Wang, Xun Qian, Fengming He, Xiyun Hu et al.UIST 2020 · 72 citations
  5. IEDS: Exploring an Intelli-Embodied Design Space Combining Designer, AR, and GAI to Support Industrial Conceptual DesignHongbo Zhang, Pei Chen, Jingwen Yang, Yifei Wu et al.CHI 2025 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines