ACL Findings2026
MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch
Nikolay Banar, Ehsan Lotfi, Jens Van Nooten, Cristina Arhiliuc, Marija Kliocaite, Walter Daelemans
Abstract
Recently, embedding resources, including models, benchmarks, and datasets, have been widely released to support a variety of languages. However, the Dutch language remains underrepresented, typically comprising only a small fraction of the published multilingual resources. To address this gap and encourage the further development of Dutch embeddings, we introduce new resources for their evaluation and generation. First, we introduce the Massive Text Embedding Benchmark for Dutch (MTEB-NL), which includes both existing Dutch datasets and newly created ones, covering a wide range of tasks. Second, we provide a training dataset compiled from available Dutch retrieval datasets, complemented with synthetic data generated by large language models to expand task coverage beyond retrieval. Finally, we release a series of E5-NL compact yet efficient embedding models that demonstrate strong performance across multiple tasks. We make our resources publicly available through the Hugging Face Hub and the MTEB package.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 1 top-tier paper that cites it · related work
Free to start. No credit card required.
Cited by top-tier papers1
Builds on18
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationJianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo et al.ACL Findings 2024 · 762 citations
- RobBERT: a Dutch RoBERTa-based Language ModelPieter Delobelle, Thomas Winters, Bettina BerendtEMNLP Findings 2020 · 248 citations
- Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the SocietyFiroj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad et al.EMNLP Findings 2021 · 133 citations
- SLURP: A Spoken Language Understanding Resource PackageEmanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, Verena RieserEMNLP 2020 · 129 citations
Related papers
- MMTEB: Massive Multilingual Text Embedding BenchmarkKenneth C. Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos et al.ICLR 2025 · 10 citations
- SkMTEB: Slovak Massive Text Embedding Benchmark and Model AdaptationMarek Suppa, Andrej Ridzik, Daniel Hládek, Natália Knazeková et al.ACL 2026
- FaMTEB: Massive Text Embedding Benchmark in Persian LanguageErfan Zinvandi, Morteza Alikhani, Mehran Sarmadi, Zahra Pourbahman et al.EMNLP Findings 2025 · 1 citation
- TR-MTEB: A Comprehensive Benchmark and Embedding Model Suite for Turkish Sentence RepresentationsMehmet Selman Baysan, Tunga GungorEMNLP Findings 2025 · 3 citations
- PL-MTEB: Polish Massive Text Embedding BenchmarkRafal Poswiata, Slawomir Dadas, Michal Wiktor PerelkiewiczACL Findings 2026
