AI Personalization Paradox: Reading Highlights for Personalized AI-Assisted Writing Increases Engagement but Undermines Autonomy and Ownership

AI-assisted writing raises concerns about autonomy and ownership when benefiting writers. Personalization has been proposed as an effective solution while also risking writers' reliance on AI and behavior shifting. For better personalization design, existing studies rely on interaction and information solely within the writing phase; however, few studies have examined how reading behaviors can inform personalized writing. This study investigates the effects of integrating reading highlights for personalization on AI-assisted writing. A between-subjects study with 46 participants revealed that the personalization condition encouraged participants to produce more highlights. However, highlighting unexpectedly shifted from a sense-making strategy to an instrumental act of "feeding the AI," leading to significant reliance on AI and declines in writers' sense of autonomy, ownership, and self-credit. These findings indicate personalization risks in AI-assisted writing, emphasize the importance of personalization strategies, and provide design implications.

• Human-centered computing → Empirical studies in HCI; Human computer interaction (HCI).