Lune

EMNLP Findings2025

Profiling LLM's Copyright Infringement Risks under Adversarial Persuasive Prompting

Jikai Long, Ming Liu, Xiusi Chen, Jialiang Xu, Shenglan Li, Zhaozhuo Xu, Denghui Zhang

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in text generation but raise concerns regarding potential copyright infringement. While prior research has explored mitigation strategies like content filtering and alignment, the impact of adversarial persuasion techniques in eliciting copyrighted content remains underexplored. This paper investigates how structured persuasion strategies, including logical appeals, emotional framing, and compliance techniques, can be used to manipulate LLM outputs and potentially increase copyright risks. We introduce a structured persuasion workflow, incorporating query mutation, intention-preserving filtering, and few-shot prompting, to systematically analyze the influence of persuasive prompts on LLM responses. Through experiments on state-of-the-art LLMs, including GPT-4o-mini and Claude-3-haiku, we quantify the effectiveness of different persuasion techniques and assess their implications for AI safety. Our results highlight the vulnerabilities of LLMs to adversarial persuasion and provide empirical evidence of the increased risk of generating copyrighted content under such influence. We conclude with recommendations for strengthening model safeguards and future directions for enhancing LLM robustness against manipulation. Code is available at https://github.com/Rongite/Persuasion.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on6

  1. Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
  2. Working With AI to Persuade: Examining a Large Language Model's Ability to Generate Pro-Vaccination MessagesElise Karinshak, Sunny Xun Liu, Joon Sung Park, Jeffrey T. HancockCSCW 2023 · 163 citations
  3. Quantifying Memorization Across Neural Language ModelsNicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee et al.ICLR 2023 · 158 citations
  4. How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMsYi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang et al.ACL 2024 · 64 citations
  5. Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information RetrievalKazuaki Furumai, Roberto Legaspi, Julio Romero, Yudai Yamazaki et al.EMNLP Findings 2024 · 9 citations

Related papers

  1. Vulnerability of LLMs' Stated Belief? LLMs Belief Resistance Check Through Strategic Persuasive Conversation InterventionsFan Huang, Haewoon Kwak, Jisun AnACL Findings 2026
  2. "Can LLMs Persuade Humans with Deception?": From a Deceptive Strategy Taxonomy to a Large-Scale Empirical StudyHaein Yeo, Seungwan Jin, Taehyung Noh, Yejin Shin et al.CHI 2026 · 2 citations
  3. SemanticCamo: Jailbreaking Large Language Models through Semantic CamouflageJihui Yan, Xiaocui Yang, Daling Wang, Shi Feng et al.ACL Findings 2025 · 3 citations
  4. Can a Large Language Model be a Gaslighter?Wei Li, Luyao Zhu, Yang Song, Ruixi Lin et al.ICLR 2025
  5. LLM Whisperer: An Inconspicuous Attack to Bias LLM ResponsesWeiran Lin, Anna Gerchanovsky, Omer Akgul, Lujo Bauer et al.CHI 2025 · 23 citations
Dusk over the sea between two cliffs drawn in fine vertical lines