Co-Writing with Opinionated Language Models Affects Users' Views
Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, Mor Naaman
Abstract
If large language models like GPT-3 preferably produce a particular point of view, they may influence people’s opinions on an unknown scale. This study investigates whether a language-model-powered writing assistant that generates some opinions more often than others impacts what users write – and what they think. In an online experiment, we asked participants (N=1,506) to write a post discussing whether social media is good for society. Treatment group participants used a language-model-powered writing assistant configured to argue that social media is good or bad for society. Participants then completed a social media attitude survey, and independent judges (N=500) evaluated the opinions expressed in their writing. Using the opinionated language model affected the opinions expressed in participants’ writing and shifted their opinions in the subsequent attitude survey. We discuss the wider implications of our results and argue that the opinions built into AI language technologies need to be monitored and engineered more carefully.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
65 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models cites The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
- Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models cites CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
- Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models cites Language Models are Few-Shot Learners
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances cites Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances cites The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances cites CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
- The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers cites Language Models are Few-Shot Learners
- Does Writing with Language Models Reduce Content Diversity? cites The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
- CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities cites The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
- CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities cites Language Models are Few-Shot Learners
- Language Models are Few-Shot Learners cites Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
- How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries cites Language Models are Few-Shot Learners
Cited by top-tier papers65
- Does Writing with Language Models Reduce Content Diversity?Vishakh Padmakumar, He HeICLR 2024 · 173 citations
- The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI RelationshipsRenwen Zhang, Han Li, Han Meng, Jinyuan Zhan et al.CHI 2025 · 122 citations
- How Knowledge Workers Think Generative AI Will (Not) Transform Their IndustriesAllison Woodruff, Renee Shelby, Patrick Gage Kelley, Steven Rousso-Schindler et al.CHI 2024 · 109 citations
- Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language ModelsParamveer S. Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub et al.CHI 2024 · 102 citations
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural NuancesDhruv Agarwal, Mor Naaman, Aditya VashisthaCHI 2025 · 93 citations
Builds on9
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
- CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model CapabilitiesMina Lee, Percy Liang, Qian YangCHI 2022 · 340 citations
- Reducing Sentiment Bias in Language Models via Counterfactual EvaluationPo-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth et al.EMNLP Findings 2020 · 124 citations
- The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English WritersDaniel Buschek, Martin Zürn, Malin EibandCHI 2021 · 106 citations
Related papers
- The Value, Benefits, and Concerns of Generative AI-Powered Assistance in WritingZhuoyan Li, Chen Liang, Jing Peng, Ming YinCHI 2024 · 78 citations
- Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated CommunicationLiye Fu, Benjamin Newman, Maurice Jakesch, Sarah KrepsCHI 2023 · 28 citations
- An Empirical Study to Understand How Students Use ChatGPT for Writing EssaysAndrew Jelson, Daniel Manesh, Alice Jang, Daniel Dunlap et al.CHI 2026 · 3 citations
- Reactive Writers: How Co-Writing with AI Changes How We Engage with IdeasAdvait Bhat, Marianne Aubin Le Quéré, Mor Naaman, Maurice JakeschCHI 2026 · 5 citations
- Inertia in Moral and Value Judgments of Large Language ModelsBruce W. Lee, Yeongheon Lee, Hyunsoo ChoACL 2026 · 5 citations
