ACL Findings2026
Position: From Noise to Signal to Selbstzweck - Reframing Human Label Variation in the Era of Post-training in NLP
Shanshan Xu, T. Y. S. S. Santosh, Barbara Plank
Abstract
Human Label Variation (HLV) refers to legitimate disagreement in annotation that reflects the diversity of human perspectives rather than mere error. Long treated in NLP as noise to be eliminated, HLV has only recently been reframed as a signal for improving model robustness. With the rise of large language models (LLMs) and post-training methods such as human feedback-based alignment, the role of HLV has become increasingly consequential. Yet current preference-learning datasets routinely collapse multiple annotations into a single label, flattening diverse perspectives into artificial consensus. Preserving HLV is necessary not only for pluralistic alignment but also for sociotechnical safety evaluation, where model behavior must be assessed in relation to human interaction and societal context. This position paper argues that preserving HLV as an embodiment of human pluralism must be treated as a Selbstzweck, an intrinsic value in itself. We analyze the limitations of existing preference datasets and propose actionable strategies for incorporating HLV into dataset construction to better preserve pluralistic human values.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on18
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
- What Can We Learn from Collective Human Opinions on Natural Language Inference Data?Yixin Nie, Xiang Zhou, Mohit BansalEMNLP 2020 · 77 citations
- Hybrid Preferences: Learning to Route Instances for Human vs. AI FeedbackLester James Validad Miranda, Yizhong Wang, Yanai Elazar, Sachin Kumar et al.ACL 2025 · 23 citations
- GoEmotions: A Dataset of Fine-Grained EmotionsDorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan S. Cowen et al.ACL 2020 · 16 citations
- Hidden Persuaders: LLMs' Political Leaning and Their Influence on VotersYujin Potter, Shiyang Lai, Junsol Kim, James Evans et al.EMNLP 2024 · 14 citations
Related papers
- Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment DatasetLily H Zhang, Smitha Milli, Karen Long Jusko, Jonathan Smith et al.ICLR 2026 · 41 citations
- Label and Explanation Variation in LLM-Based Annotation: a Case Study in Natural Language InferenceArtur Kulmizev, Erika Lombart, Patrick Watrin, Marie-Catherine de MarneffeACL 2026
- PLLuM-Align: Polish Preference Dataset for Large Language Model AlignmentKarolina Seweryn, Anna Kolos, Agnieszka Karlinska, Katarzyna Lorenc et al.EMNLP 2025
- Strong Preferences Affect the Robustness of Preference Models and Value AlignmentZiwei Xu, Mohan S. KankanhalliICLR 2025
- When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference LearningYijiang River Dong, Tiancheng Hu, Yinhong Liu, Ahmet Üstün et al.EMNLP Findings 2025
