Lune

ACL Findings2026

Position: From Noise to Signal to Selbstzweck - Reframing Human Label Variation in the Era of Post-training in NLP

Shanshan Xu, T. Y. S. S. Santosh, Barbara Plank

Abstract

Human Label Variation (HLV) refers to legitimate disagreement in annotation that reflects the diversity of human perspectives rather than mere error. Long treated in NLP as noise to be eliminated, HLV has only recently been reframed as a signal for improving model robustness. With the rise of large language models (LLMs) and post-training methods such as human feedback-based alignment, the role of HLV has become increasingly consequential. Yet current preference-learning datasets routinely collapse multiple annotations into a single label, flattening diverse perspectives into artificial consensus. Preserving HLV is necessary not only for pluralistic alignment but also for sociotechnical safety evaluation, where model behavior must be assessed in relation to human interaction and societal context. This position paper argues that preserving HLV as an embodiment of human pluralism must be treated as a Selbstzweck, an intrinsic value in itself. We analyze the limitations of existing preference datasets and propose actionable strategies for incorporating HLV into dataset construction to better preserve pluralistic human values.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters cites Whose Opinions Do Language Models Reflect?
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
  2. What Can We Learn from Collective Human Opinions on Natural Language Inference Data?Yixin Nie, Xiang Zhou, Mohit BansalEMNLP 2020 · 77 citations
  3. Hybrid Preferences: Learning to Route Instances for Human vs. AI FeedbackLester James Validad Miranda, Yizhong Wang, Yanai Elazar, Sachin Kumar et al.ACL 2025 · 23 citations
  4. GoEmotions: A Dataset of Fine-Grained EmotionsDorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan S. Cowen et al.ACL 2020 · 16 citations
  5. Hidden Persuaders: LLMs' Political Leaning and Their Influence on VotersYujin Potter, Shiyang Lai, Junsol Kim, James Evans et al.EMNLP 2024 · 14 citations

Related papers

  1. Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment DatasetLily H Zhang, Smitha Milli, Karen Long Jusko, Jonathan Smith et al.ICLR 2026 · 41 citations
  2. Label and Explanation Variation in LLM-Based Annotation: a Case Study in Natural Language InferenceArtur Kulmizev, Erika Lombart, Patrick Watrin, Marie-Catherine de MarneffeACL 2026
  3. PLLuM-Align: Polish Preference Dataset for Large Language Model AlignmentKarolina Seweryn, Anna Kolos, Agnieszka Karlinska, Katarzyna Lorenc et al.EMNLP 2025
  4. Strong Preferences Affect the Robustness of Preference Models and Value AlignmentZiwei Xu, Mohan S. KankanhalliICLR 2025
  5. When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference LearningYijiang River Dong, Tiancheng Hu, Yinhong Liu, Ahmet Üstün et al.EMNLP Findings 2025
Dusk over the sea between two cliffs drawn in fine vertical lines