Feeling Right vs. Being Right: How AI Sycophancy Affects Value-Laden Deliberation
Jeongwoo Ryu, Soomin Kim, Jinsu Eun, Kyusik Kim, Changhoon Oh, Bongwon Suh
摘要
As people increasingly turn to AI for personal deliberation beyond task-oriented assistance, concerns about sycophancy in these valueladen contexts have grown. Unlike human flattery, which is intentional and self-interested, AI sycophancy emerges as a byproduct of RLHF's reward structure for user-preference alignment. Yet the observable behavior is similar: both produce responses that preserve what users want to hear. Focusing on this phenomenon through Goffman's face-work framework, we operationalize AI sycophancy as excessive facesaving, either active (preserving positive face through agreement) or passive (preserving negative face by withholding challenge). In a mixed-methods study (N = 31), participants engaged with AI across three moral dilemmas under these conditions and a non-sycophantic neutral baseline. Sycophantic responses increased decision confidence but reduced openminded thinking; participants felt supported yet found the conversations unproductive. Neutral responses, though initially uncomfortable, promoted cognitive flexibility and meaningful deliberation. These findings reveal a confidencecompetence trade-off in AI-mediated moral reasoning and suggest that effective AI for personal deliberation requires calibrated friction, not unconditional agreement.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingZana Buçinca, Maja Barbara Malaya, Krzysztof Z. GajosCSCW 2021 · 被引用 962 次
- Towards Understanding Sycophancy in Language ModelsMrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud 等ICLR 2024 · 被引用 762 次
- Design Principles for Generative AI ApplicationsJustin D. Weisz, Jessica He, Michael J. Muller, Gabriela Hoefer 等CHI 2024 · 被引用 221 次
- Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information SeekingNikhil Sharma, Q. Vera Liao, Ziang XiaoCHI 2024 · 被引用 123 次
相关 Paper
- Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving TasksJessica Y. Bo, Majeed Kazemitabaar, Mengqing Deng, Michael Inzlicht 等CHI 2026 · 被引用 8 次
- Does Sycophancy Change Decisions? Effect of LLM Sycophancy on AI-Assisted Decision-MakingZejian Li, Jiaman Pan, Qi Liu, Yuning Xi 等CHI 2026 · 被引用 1 次
- Be Friendly, Not Friends: How LLM Sycophancy Shapes User TrustYuan Sun, Ting WangCHI 2026 · 被引用 15 次
- Cognitive models can reveal interpretable value trade-offs in language modelsSonia Krishna Murthy, Rosie Zhao, Jennifer Hu, Sham M. Kakade 等ICLR 2026 · 被引用 2 次
- ELEPHANT: Measuring and understanding social sycophancy in LLMsMyra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe 等ICLR 2026
