Lune

EMNLP Findings2023

Enhancing Neural Machine Translation with Semantic Units

Langlin Huang, Shuhao Gu, Zhuocheng Zhang, Yang Feng

Abstract

Conventional neural machine translation (NMT) models typically use subwords and words as the basic units for model input and comprehension. However, complete words and phrases composed of several tokens are often the fundamental units for expressing semantics, referred to as semantic units. To address this issue, we propose a method Semantic Units for Machine Translation (SU4MT) which models the integral meanings of semantic units within a sentence, and then leverages them to provide a new perspective for understanding the sentence. Specifically, we first propose Word Pair Encoding (WPE), a phrase extraction method to help identify the boundaries of semantic units. Next, we design an Attentive Semantic Fusion (ASF) layer to integrate the semantics of multiple subwords into a single vector: the semantic unit representation. Lastly, the semantic-unit-level sentence representation is concatenated to the token-level one, and they are combined as the input of encoder. Experimental results demonstrate that our method effectively models and leverages semantic-unit-level information and outperforms the strong baselines. The code is available at https://github.com/ictnlp/SU4MT.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining cites LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on7

  1. LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentionIkuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda et al.EMNLP 2020 · 562 citations
  2. Decomposed Meta-Learning for Few-Shot Named Entity RecognitionTingting Ma, Huiqiang Jiang, Qianhui Wu, Tiejun Zhao et al.ACL Findings 2022 · 102 citations
  3. UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic MiningJiacheng Li, Jingbo Shang, Julian J. McAuleyACL 2022 · 68 citations
  4. BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
  5. Learning Source Phrase Representations for Neural Machine TranslationHongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu et al.ACL 2020 · 18 citations

Related papers

  1. A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine TranslationYongjing Yin, Fandong Meng, Jinsong Su, Chulun Zhou et al.ACL 2020 · 145 citations
  2. Incorporating BERT into Neural Machine TranslationJinhua Zhu, Yingce Xia, Lijun Wu, Di He et al.ICLR 2020 · 391 citations
  3. Uncertainty-Aware Semantic Augmentation for Neural Machine TranslationXiangpeng Wei, Heng Yu, Yue Hu, Rongxiang Weng et al.EMNLP 2020 · 20 citations
  4. Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence GenerationFrancois Meyer, Jan BuysACL Findings 2023 · 3 citations
  5. SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine TranslationShuo Ren, Long Zhou, Shujie Liu, Furu Wei et al.ACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines