CROSSNEWS: A Cross-Genre Authorship Verification and Attribution Benchmark
Marcus Ma, Duong Minh Le, Junmo Kang, Yao Dou, John Cadigan, Dayne Freitag, Alan Ritter, Wei Xu
Abstract
Authorship models have historically generalized poorly to new domains because of the wide distribution of author-identifying signals across domains. In particular, the effects of topic and genre are highly domain-dependent and impact authorship analysis performance greatly. This paper addresses the existing data gap in authorship for these resources by introducing CROSSNEWS, a novel cross-genre dataset that connects formal journalistic articles and casual social media posts. CROSSNEWS is the largest authorship dataset of its kind for supporting both verification and attribution tasks, with comprehensive topic and genre annotations. We use CROSSNEWS to demonstrate that current models exhibit poor performance in genre transfer scenarios, underscoring the need for authorship models robust to genre-specific effects. We also explore SELMA, a new LLM embedding approach for large-scale authorship setups that outperforms existing models in both same-genre and cross-genre settings.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Builds on4
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Can Large Language Models Identify Authorship?Baixiang Huang, Canyu Chen, Kai ShuEMNLP Findings 2024 · 30 citations
- Who Wrote it and Why? Prompting Large-Language Models for Authorship VerificationChia-Yu Hung, Zhiqiang Hu, Yujia Hu, Roy Ka-Wei LeeEMNLP Findings 2023 · 10 citations
- Improving Text Embeddings with Large Language ModelsLiang Wang, Nan Yang, Xiaolong Huang, Linjun Yang et al.ACL 2024
Related papers
- The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship VerificationAbraham Israeli, Shuai Liu, Jonathan May, David JurgensACL Findings 2025
- The Topic Confusion Task: A Novel Evaluation Scenario for Authorship AttributionMalik H. Altakrori, Jackie Chi Kit Cheung, Benjamin C. M. FungEMNLP Findings 2021 · 11 citations
- Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social MediaShubham Mittal, Megha Sundriyal, Preslav NakovEMNLP 2023 · 4 citations
- A Bayesian Approach to Harnessing the Power of LLMs in Authorship AttributionZhengmian Hu, Tong Zheng, Heng HuangEMNLP 2024 · 1 citation
- STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence RetrievalCostanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou et al.EMNLP Findings 2020 · 12 citations
