Lune

ACL Findings2022

Fast Nearest Neighbor Machine Translation

Yuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu, Xiaofei Sun, Tianwei Zhang, Jiwei Li

Abstract

Though nearest neighbor Machine Translation (kNN-MT) (Khandelwal et al., 2020) has proved to introduce significant performance boosts over standard neural MT systems, it is prohibitively slow since it uses the entire reference corpus as the datastore for the nearest neighbor search. This means each step for each beam in the beam search has to search over the entire reference corpus. kNN-MT is thus two-orders slower than vanilla MT models, making it hard to be applied to real-world applications, especially online services. In this work, we propose Fast kNN-MT to address this issue. Fast kNN-MT constructs a significantly smaller datastore for the nearest neighbor search: for each word in a source sentence, Fast kNN-MT first selects its nearest tokenlevel neighbors, which is limited to tokens that are the same as the query token. Then at each decoding step, in contrast to using the entire corpus as the datastore, the search space is limited to target tokens corresponding to the previously selected reference source tokens. This strategy avoids search through the whole datastore for nearest neighbors and drastically improves decoding efficiency. Without loss of performance, Fast kNN-MT is two-orders faster than kNN-MT, and is only two times slower than the standard NMT model. Fast kNN-MT enables the practical use of kNN-MT systems in real-world MT applications. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

17 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine Translation cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine Translation cites Generalization through Memorization: Nearest Neighbor Language Models
  • GNN-LM: Language Modeling based on Global Contexts via GNN cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • GNN-LM: Language Modeling based on Global Contexts via GNN cites Generalization through Memorization: Nearest Neighbor Language Models
  • GNN-LM: Language Modeling based on Global Contexts via GNN cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models cites Generalization through Memorization: Nearest Neighbor Language Models
  • Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models cites Retrieval Augmented Language Model Pre-Training
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Generalization through Memorization: Nearest Neighbor Language Models
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Retrosynthesis Prediction with Local Template Retrieval cites Generalization through Memorization: Nearest Neighbor Language Models
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks cites Generalization through Memorization: Nearest Neighbor Language Models
  • Retrieval Augmented Language Model Pre-Training cites Generalization through Memorization: Nearest Neighbor Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers17

  1. Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language ModelsWenqi Jiang, Marco Zeller, Roger Waleffe, Torsten Hoefler et al.VLDB 2025 · 50 citations
  2. GNN-LM: Language Modeling based on Global Contexts via GNNYuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun et al.ICLR 2022 · 46 citations
  3. Retrosynthesis Prediction with Local Template RetrievalShufang Xie, Rui Yan, Junliang Guo, Yingce Xia et al.AAAI 2023 · 20 citations
  4. Towards Robust k-Nearest-Neighbor Machine TranslationHui Jiang, Ziyao Lu, Fandong Meng, Chulun Zhou et al.EMNLP 2022 · 16 citations
  5. kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine TranslationShudong Liu, Xuebo Liu, Derek F. Wong, Zhaocong Li et al.ACL 2023 · 14 citations

Builds on18

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  3. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  4. On Layer Normalization in the Transformer ArchitectureRuibin Xiong, Yunchang Yang, Di He, Kai Zheng et al.ICML 2020 · 1,388 citations
  5. Generalization through Memorization: Nearest Neighbor Language ModelsUrvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2020 · 1,038 citations

Related papers

  1. Subset Retrieval Nearest Neighbor Machine TranslationHiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama et al.ACL 2023 · 7 citations
  2. Simple and Scalable Nearest Neighbor Machine TranslationYuhan Dai, Zhirui Zhang, Qiuzhi Liu, Qu Cui et al.ICLR 2023 · 9 citations
  3. Chunk-based Nearest Neighbor Machine TranslationPedro Henrique Martins, Zita Marinho, André F. T. MartinsEMNLP 2022 · 17 citations
  4. Efficient Cluster-Based k-Nearest-Neighbor Machine TranslationDexin Wang, Kai Fan, Boxing Chen, Deyi XiongACL 2022 · 35 citations
  5. Nearest Neighbor Machine TranslationUrvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2021 · 323 citations
Dusk over the sea between two cliffs drawn in fine vertical lines