Lune

ACL2024Top-tier venue

M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Fikri Aji, Nizar Habash

Abstract

The advent of Large Language Models (LLMs) has brought an unprecedented surge in machinegenerated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. The need to identify and differentiate such content from genuine human-generated text is critical in combating disinformation, preserving the integrity of education and scientific fields, and maintaining trust in communication. In this work, we address this problem by introducing a new benchmark based on a multilingual, multidomain, and multi-generator corpus of MGTs -M4GT-Bench. The benchmark is compiled of three tasks: (1) mono-lingual and multi-lingual binary MGT detection; (2) multi-way detection where one need to identify, which particular model generated the text; and (3) mixed humanmachine text detection, where a word boundary delimiting MGT from human-written content should be determined. On the developed benchmark, we have tested several MGT detection baselines and also conducted an evaluation of human performance. We see that obtaining good performance in MGT detection usually requires an access to the training data from the same domain and generators. The benchmark is available at https://github. com/mbzuai-nlp/M4GT-Bench. Source Human Parallel Data Total New test Domain Total= Upsample+ Parallel davinci-003 ChatGPT Cohere Dolly-v2 BLOOMz Machine GPT-4 OUTFOX 16

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

9 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective cites DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
  • TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices cites DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
  • TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices cites Unsupervised Cross-lingual Representation Learning at Scale
  • Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense cites DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
  • DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature cites Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
  • Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense cites A Watermark for Large Language Models
  • DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning cites Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
  • DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning cites DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
  • DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning cites A Watermark for Large Language Models
  • Authorship Attribution in Multilingual Machine-Generated Texts cites DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
  • Authorship Attribution in Multilingual Machine-Generated Texts cites Unsupervised Cross-lingual Representation Learning at Scale
  • Authorship Attribution in Multilingual Machine-Generated Texts cites A Watermark for Large Language Models
  • DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature cites A Watermark for Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers9

  1. DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation LearningYongxin He, Shan Zhang, Yixuan Cao, Lei Ma et al.NeurIPS 2025 · 13 citations
  2. Authorship Attribution in Multilingual Machine-Generated TextsLucio La Cava, Dominik Macko, Róbert Móro, Ivan Srba et al.ACL 2026 · 7 citations
  3. Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AIYuxia Wang, Rui Xing, Jonibek Mansurov, Giovanni Puccetti et al.ACL 2026 · 3 citations
  4. TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing PracticesGerrit Quaremba, Elizabeth Black, Denny Vrandecic, Elena SimperlICLR 2026 · 2 citations
  5. On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty PerspectiveYikai Guo, Bin Wang, Xilai Fan, Wenjun Ke et al.ICML 2026

Builds on20

  1. Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
  2. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability CurvatureEric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning et al.ICML 2023 · 988 citations
  3. A Watermark for Large Language ModelsJohn Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz et al.ICML 2023 · 854 citations
  4. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defenseKalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting et al.NeurIPS 2023 · 657 citations
  5. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations

Related papers

  1. MGTBench: Benchmarking Machine-Generated Text DetectionXinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes et al.CCS 2024 · 30 citations
  2. MAGE: Machine-generated Text Detection in the WildYafu Li, Qintong Li, Leyang Cui, Wei Bi et al.ACL 2024 · 44 citations
  3. Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement MeasurementZihao Cheng, Li Zhou, Feng Jiang, Benyou Wang et al.WWW 2025 · 20 citations
  4. OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and AttributionLucio La Cava, Andrea TagarelliEMNLP 2025
  5. MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection BenchmarkDominik Macko, Róbert Móro, Adaku Uchendu, Jason Samuel Lucas et al.EMNLP 2023 · 25 citations
Dusk over the sea between two cliffs drawn in fine vertical lines