Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection

The rise of Large Language Models (LLMs) necessitates accurate AI-generated text detection. However, current approaches largely overlook the influence of author characteristics. We investigate how sociolinguistic attributes-gender, CEFR proficiency, academic field, and language environment-impact stateof-the-art AI text detectors. Using the ICNALE corpus of human-authored texts and parallel AIgenerated texts from diverse LLMs, we conduct a rigorous evaluation employing multi-factor ANOVA and weighted least squares (WLS). Our results reveal significant biases: CEFR proficiency and language environment consistently affected detector accuracy, while gender and academic field showed detector-dependent effects. These findings highlight the crucial need for socially aware AI text detection to avoid unfairly penalizing specific demographic groups. We offer novel empirical evidence, a robust statistical framework, and actionable insights for developing more equitable and reliable detection systems in real-world, out-of-domain contexts. This work paves the way for future research on bias mitigation, inclusive evaluation benchmarks, and socially responsible LLM detectors. Detector CEFR Sex Academic Genre Language Env. binoculars Yes No No Yes chatgptroberta Yes No No No detectgpt Yes No Yes Yes fastdetectgpt Yes No No Yes fastdetectllm Yes No Yes Yes gltr Yes No No Yes gpt2-base Yes No Yes Yes gpt2-large Yes No Yes Yes llmdet Yes No Yes Yes radar Yes No No Yes