Lune

EMNLP Findings2025

Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation

Evgeniia Tokarchuk, Sergey Troshin, Vlad Niculae

Abstract

Augmenting neural machine translation with external memory at decoding time, in the form of k-nearest neighbors machine translation (k-NN MT), is a well-established strategy for increasing translation performance. k-NN MT retrieves a set of tokens that occurred in the most similar contexts recorded in a prepared data store, using hidden state representations of translation contexts as vector lookup keys. One of the main disadvantages of this method is the high computational cost and memory requirements. Since an exhaustive search is not feasible in large data stores, practitioners commonly use approximate k-NN lookup, yet even such algorithms are a bottleneck. In contrast to research directions seeking to accelerate k-NN MT by reducing data store size or the number of lookup calls, we pursue an orthogonal direction based on the performance properties of approximate k-NN lookup data structures.

In particular, we propose to encourage angular dispersion of the neural hidden representations of contexts. We show that improving dispersion leads to better balance in the retrieval data structures, accelerating retrieval and slightly improving translations.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Fast Nearest Neighbor Machine Translation cites Nearest Neighbor Machine Translation
  • Efficient Cluster-Based k-Nearest-Neighbor Machine Translation cites Nearest Neighbor Machine Translation
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on15

  1. Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
  2. Nearest Neighbor Machine TranslationUrvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2021 · 323 citations
  3. Efficient Cluster-Based k-Nearest-Neighbor Machine TranslationDexin Wang, Kai Fan, Boxing Chen, Deyi XiongACL 2022 · 35 citations
  4. Fast Nearest Neighbor Machine TranslationYuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu et al.ACL Findings 2022 · 32 citations
  5. MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal AnglesZhennan Wang, Canqun Xiang, Wenbin Zou, Chen XuNeurIPS 2020 · 25 citations

Related papers

  1. Subset Retrieval Nearest Neighbor Machine TranslationHiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama et al.ACL 2023 · 7 citations
  2. Simple and Scalable Nearest Neighbor Machine TranslationYuhan Dai, Zhirui Zhang, Qiuzhi Liu, Qu Cui et al.ICLR 2023 · 9 citations
  3. Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain AdaptationAbudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula et al.ACL 2025
  4. Revisiting Source Context in Nearest Neighbor Machine TranslationXuanhong Li, Peng Li, Po HuEMNLP 2023
  5. Why do Nearest Neighbor Language Models Work?Frank F. Xu, Uri Alon, Graham NeubigICML 2023 · 33 citations
Dusk over the sea between two cliffs drawn in fine vertical lines