EMNLP Findings2021
'Just What do You Think You're Doing, Dave?' A Checklist for Responsible Data Use in NLP
Anna Rogers, Timothy Baldwin, Kobi Leins
Abstract
A key part of the NLP ethics movement is responsible use of data, but exactly what that means or how it can be best achieved remain unclear. This position paper discusses the core legal and ethical principles for collection and sharing of textual data, and the tensions between them.
We propose a potential checklist for responsible data (re-)use that could both standardise the peer review of conference submissions, as well as enable a more in-depth view of published research across the community. Our proposal aims to contribute to the development of a consistent standard for data (re-)use, embraced across NLP conferences.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 11 top-tier papers that cite it · related work
Free to start. No credit card required.
Cited by top-tier papers11
- Reproducibility in Computational Linguistics: Is Source Code Enough?Mohammad Arvan, Luís Pina, Natalie PardeEMNLP 2022 · 12 citations
- Building Better: Avoiding Pitfalls in Developing Language Resources when Data is ScarceNedjma Ousidhoum, Meriem Beloucif, Saif M. MohammadACL 2025 · 5 citations
- Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science ResearchYida Mu, Mali Jin, Xingyi Song, Nikolaos AletrasEMNLP 2024 · 4 citations
- ECBD: Evidence-Centered Benchmark Design for NLPYu Lu Liu, Su Lin Blodgett, Jackie C. K. Cheung, Vera Liao et al.ACL 2024 · 3 citations
- In Benchmarks We Trust ... Or Not?Ine Gevers, Victor De Marez, Jens Van Nooten, Jens Lemmens et al.EMNLP 2025 · 1 citation
Builds on5
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Extracting Training Data from Large Language ModelsNicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski et al.USENIX Security 2021 · 2,866 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
- Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current PracticesSebastin Santy, Anku Rani, Monojit ChoudhuryACL Findings 2021 · 9 citations
- Changing the World by Changing the DataAnna RogersACL 2021
Related papers
- Thorny Roses: Investigating the Dual Use Dilemma in Natural Language ProcessingLucie-Aimée Kaffee, Arnav Arora, Zeerak Talat, Isabelle AugensteinEMNLP Findings 2023 · 12 citations
- Reproducibility in NLP: What Have We Learned from the Checklist?Ian Magnusson, Noah A. Smith, Jesse DodgeACL Findings 2023 · 14 citations
- Is NLP Ready for Standardization?Lauriane AufrantEMNLP Findings 2022 · 4 citations
- Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and BeyondNils Dycke, Ilia Kuznetsov, Iryna GurevychEMNLP Findings 2022 · 10 citations
- On the Gap between Adoption and Understanding in NLPFederico Bianchi, Dirk HovyACL Findings 2021 · 25 citations
