AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive Contexts

Distinguishing LLM-generated text from human-written is a key challenge for safe and ethical NLP, particularly in high-stake settings such as persuasive online discourse. While recent work focuses on detection, real-world use cases also demand interpretable tools to help humans understand and distinguish LLMgenerated texts. To this end, we present an analysis framework comparing human-and LLM-authored arguments using two easilyinterpretable feature sets: general-purpose linguistic features (e.g., lexical richness, syntactic complexity) and domain-specific features related to argument quality (e.g., logical soundness, engagement strategies). Applied to /r/ChangeMyView arguments by humans and three LLMs, our method reveals clear patterns: LLM-generated counter-arguments show lower type-token and lemma-token ratios but higher emotional intensity -particularly in anticipation and trust. They more closely resemble textbook-quality arguments -cogent, justified, explicitly respectful toward others, and positive in tone. Moreover, counter-arguments generated by LLMs converge more closely with the original post's style and quality than those written by humans. Finally, we demonstrate that these differences enable a lightweight, interpretable, and highly effective classifier for detecting LLM-generated comments in CMV. Milad Alshomary and Henning Wachsmuth. 2023. Conclusion-based counter-argument generation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 957-967, Dubrovnik, Croatia. Association for Computational Linguistics.