EMNLP Findings2024
Finding the Optimal Byte-Pair Encoding Merge Operations for Neural Machine Translation in a Low-Resource Setting
Kristine Mae M. Adlaon, Nelson Marcos
Abstract
This paper investigates the impact of different Byte Pair Encoding (BPE) configurations, specifically, merge operations on neural machine translation (NMT) performance for the Filipino-Cebuano language pair across various text domains. Results demonstrate that smaller BPE configurations, notably 2k, 5k, and 8k consistently yield higher BLEU scores, indicating improved translation quality through finer tokenization granularity. Conversely, larger BPE configurations and the absence of BPE result in lower BLEU scores, suggesting a decline in translation quality due to coarser tokenization. Additionally, these findings help us understand how the size of the model and how finely we break down words affect the quality of translations. This knowledge will be useful for improving translation systems, especially for languages that don't have many parallel texts available for training.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on3
- Finding the Optimal Vocabulary Size for Neural Machine TranslationThamme Gowda, Jonathan MayEMNLP Findings 2020 · 44 citations
- Dynamic Programming Encoding for Subword Segmentation in Neural Machine TranslationXuanli He, Gholamreza Haffari, Mohammad NorouziACL 2020 · 33 citations
- Fast WordPiece TokenizationXinying Song, Alex Salcianu, Yang Song, Dave Dopson et al.EMNLP 2021
Related papers
- Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language ModelsTomohiro Sawada, Kartik GoyalEMNLP 2025
- MorphBPE: Morphology-Aware Tokenization for Efficient LLM TrainingEhsaneddin Asgari, Yassine El Kheir, MohammadAli SadraeiJavaheri, Ali NazariACL Findings 2026
- Incremental BPE TokenizationShenghu Jiang, Ruihao GongICML 2026 · 12 citations
- BPE-Dropout: Simple and Effective Subword RegularizationIvan Provilkov, Dmitrii Emelianenko, Elena VoitaACL 2020 · 17 citations
- BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer TrainingPavel Chizhov, Catherine Arnett, Elizaveta Korotkova, Ivan P. YamshchikovEMNLP 2024 · 1 citation
