Principles of Safe AI Companions for Youth: Parent and Expert Perspectives
Yaman Yu, Mohi, Aishi Debroy, Xin Cao, Karen Rudolph, Yang Wang
摘要
AI companions are increasingly popular among teenagers, yet current platforms lack safeguards to address developmental risks and harmful normalization. Despite growing concerns, little is known about how parents and developmental psychology experts assess these interactions or what protections they consider necessary. We conducted 26 semi-structured interviews with parents and experts, who reviewed real-world youth–AI companion conversation snippets. We found that stakeholders assessed risks contextually, attending to factors such as youth maturity, AI character age, and how AI characters modeled values and norms. We also identified distinct logics of assessment: parents flagged single events, such as a mention of suicide or flirtation, as high risk, whereas experts looked for patterns over time, such as repeated references to self-harm or sustained dependence. Both groups proposed interventions, with parents favoring broader oversight and experts preferring cautious, crisis-only escalation paired with youth-facing safeguards. These findings provide directions for embedding safety into AI companion design.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit NarrativesMohammad (Matt) Namvarpour, Brandon Brofsky, Jessica Y. Medina, Mamtaj Akter 等CHI 2026 · 被引用 19 次
- Do Teachers Dream of GenAI Widening Educational (In)equality? Envisioning the Future of K-12 GenAI Education from Global Teachers' PerspectivesRuiwei Xiao, Qing Xiao, Xinying Hou, Phenyo Phemelo Moletsane 等CHI 2026 · 被引用 4 次
它引用的顶会 Paper14
- The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI RelationshipsRenwen Zhang, Han Li, Han Meng, Jinyuan Zhan 等CHI 2025 · 被引用 122 次
- From Parental Control to Joint Family Oversight: Can Parents and Teens Manage Mobile Online Safety and Privacy as Equals?Mamtaj Akter, Amy J. Godfrey, Jess Kropczynski, Heather Richter Lipford 等CSCW 2022 · 被引用 64 次
- Protection or Punishment? Relating the Design Space of Parental Control Apps and Perceptions about Them to Support Parenting for Online SafetyGe Wang, Jun Zhao, Max Van Kleek, Nigel ShadboltCSCW 2021 · 被引用 63 次
- The Role of AI in Peer Support for Young People: A Study of Preferences for Human- and AI-Generated ResponsesJordyn Young, Laala M. Jawara, Diep N. Nguyen, Brian Daly 等CHI 2024 · 被引用 58 次
- Child Safety in the Smart Home: Parents' Perceptions, Needs, and Mitigation StrategiesKaiwen Sun, Yixin Zou, Jenny S. Radesky, Christopher Brooks 等CSCW 2021 · 被引用 51 次
相关 Paper
- Exploring Parent-Child Perceptions on Safety in Generative AI: Concerns, Mitigation Strategies, and Design ImplicationsYaman Yu, Tanusree Sharma, Melinda Hu, Justin Wang 等S&P 2025
- "Pikachu would electrocute people who are misbehaving": Expert, Guardian and Child Perspectives on Automated Embodied Moderators for Safeguarding Children in Social Virtual RealityCristina Fiani, Robin Bretin, Shaun Alexander Macdonald, Mohamed Khamis 等CHI 2024 · 被引用 18 次
- Who Gets to Define Safety? A Systematic Review of How Generative AI Research Addresses Youth Online SafetyOzioma Collins Oguine, Adriana Alvarado Garcia, Michael J. Muller, Karla Badillo-UrquiolaCHI 2026 · 被引用 2 次
- Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated ProbesJohn Driscoll, Yulin Chen, Viki Shi, Izak Vucharatavintara 等CHI 2026 · 被引用 2 次
- YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language ModelsYaman Yu, Yiren Liu, Yuqi Zhang, Yun Huang 等CCS 2025 · 被引用 1 次
