EMNLP Findings2025
Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation
Abstract
Augmenting neural machine translation with external memory at decoding time, in the form of k-nearest neighbors machine translation (k-NN MT), is a well-established strategy for increasing translation performance. k-NN MT retrieves a set of tokens that occurred in the most similar contexts recorded in a prepared data store, using hidden state representations of translation contexts as vector lookup keys. One of the main disadvantages of this method is the high computational cost and memory requirements. Since an exhaustive search is not feasible in large data stores, practitioners commonly use approximate k-NN lookup, yet even such algorithms are a bottleneck. In contrast to research directions seeking to accelerate k-NN MT by reducing data store size or the number of lookup calls, we pursue an orthogonal direction based on the performance properties of approximate k-NN lookup data structures.
In particular, we propose to encourage angular dispersion of the neural hidden representations of contexts. We show that improving dispersion leads to better balance in the retrieval data structures, accelerating retrieval and slightly improving translations.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on15
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
- Nearest Neighbor Machine TranslationUrvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2021 · 323 citations
- Efficient Cluster-Based k-Nearest-Neighbor Machine TranslationDexin Wang, Kai Fan, Boxing Chen, Deyi XiongACL 2022 · 35 citations
- Fast Nearest Neighbor Machine TranslationYuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu et al.ACL Findings 2022 · 32 citations
- MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal AnglesZhennan Wang, Canqun Xiang, Wenbin Zou, Chen XuNeurIPS 2020 · 25 citations
Related papers
- Subset Retrieval Nearest Neighbor Machine TranslationHiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama et al.ACL 2023 · 7 citations
- Simple and Scalable Nearest Neighbor Machine TranslationYuhan Dai, Zhirui Zhang, Qiuzhi Liu, Qu Cui et al.ICLR 2023 · 9 citations
- Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain AdaptationAbudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula et al.ACL 2025
- Revisiting Source Context in Nearest Neighbor Machine TranslationXuanhong Li, Peng Li, Po HuEMNLP 2023
- Why do Nearest Neighbor Language Models Work?Frank F. Xu, Uri Alon, Graham NeubigICML 2023 · 33 citations
