Lune

EMNLP Findings2024

Finding the Optimal Byte-Pair Encoding Merge Operations for Neural Machine Translation in a Low-Resource Setting

Kristine Mae M. Adlaon, Nelson Marcos

Abstract

This paper investigates the impact of different Byte Pair Encoding (BPE) configurations, specifically, merge operations on neural machine translation (NMT) performance for the Filipino-Cebuano language pair across various text domains. Results demonstrate that smaller BPE configurations, notably 2k, 5k, and 8k consistently yield higher BLEU scores, indicating improved translation quality through finer tokenization granularity. Conversely, larger BPE configurations and the absence of BPE result in lower BLEU scores, suggesting a decline in translation quality due to coarser tokenization. Additionally, these findings help us understand how the size of the model and how finely we break down words affect the quality of translations. This knowledge will be useful for improving translation systems, especially for languages that don't have many parallel texts available for training.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on3

  1. Finding the Optimal Vocabulary Size for Neural Machine TranslationThamme Gowda, Jonathan MayEMNLP Findings 2020 · 44 citations
  2. Dynamic Programming Encoding for Subword Segmentation in Neural Machine TranslationXuanli He, Gholamreza Haffari, Mohammad NorouziACL 2020 · 33 citations
  3. Fast WordPiece TokenizationXinying Song, Alex Salcianu, Yang Song, Dave Dopson et al.EMNLP 2021

Related papers

  1. Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language ModelsTomohiro Sawada, Kartik GoyalEMNLP 2025
  2. MorphBPE: Morphology-Aware Tokenization for Efficient LLM TrainingEhsaneddin Asgari, Yassine El Kheir, MohammadAli SadraeiJavaheri, Ali NazariACL Findings 2026
  3. Incremental BPE TokenizationShenghu Jiang, Ruihao GongICML 2026 · 12 citations
  4. BPE-Dropout: Simple and Effective Subword RegularizationIvan Provilkov, Dmitrii Emelianenko, Elena VoitaACL 2020 · 17 citations
  5. BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer TrainingPavel Chizhov, Catherine Arnett, Elizaveta Korotkova, Ivan P. YamshchikovEMNLP 2024 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines