Lune

ACL Findings2021

Perceptual Models of Machine-Edited Text

Elizabeth M. Merkhofer, Monica-Ann Mendoza, Rebecca Marvin, John C. Henderson

Abstract

We introduce a novel dataset of human judgments of machine-edited text and initial models of those perceptions. Six machine-editing methods ranging from character swapping to variational autoencoders are applied to collections of English-language social media text and scientific abstracts. The edits are judged in context for detectability and the extent to which they preserve the meaning of the original. Automated measures of semantic similarity and fluency are evaluated individually and combined to produce composite models of human perception. Both meaning preservation and detectability are predicted within 6% of the upper bound of human consensus labeling.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Automatic Detection of Generated Text is Easiest when Humans are Fooled cites BERTScore: Evaluating Text Generation with BERT
  • BLEURT: Learning Robust Metrics for Text Generation cites BERTScore: Evaluating Text Generation with BERT
  • Masked Language Model Scoring cites BERTScore: Evaluating Text Generation with BERT
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on6

  1. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  2. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
  3. Masked Language Model ScoringJulian Salazar, Davis Liang, Toan Q. Nguyen, Katrin KirchhoffACL 2020 · 167 citations
  4. BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
  5. Automatic Detection of Generated Text is Easiest when Humans are FooledDaphne Ippolito, Daniel Duckworth, Chris Callison-Burch, Douglas EckACL 2020 · 21 citations

Related papers

  1. EditLens: Quantifying the Extent of AI Editing in TextKatherine Thai, Bradley Emi, Elyas Masrour, Mohit IyyerICLR 2026 · 30 citations
  2. Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal StyleConnor Baumler, Calvin Bao, Huy Nghiem, Xinchen Yang et al.ACL 2026 · 1 citation
  3. Mind the Style Gap: Meta-Evaluation of Style and Attribute Transfer MetricsAmalie Brogaard Pauli, Isabelle Augenstein, Ira AssentEMNLP Findings 2025
  4. Raidar: geneRative AI Detection viA RewritingChengzhi Mao, Carl Vondrick, Hao Wang, Junfeng YangICLR 2024 · 66 citations
  5. How Large Language Models are Transforming Machine-Paraphrase PlagiarismJan Philip Wahle, Terry Ruas, Frederic Kirstein, Bela GippEMNLP 2022 · 26 citations
Dusk over the sea between two cliffs drawn in fine vertical lines