Lune

EMNLP Findings2022

Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering

Chun-Hin Tse, Ester Leung, William K. Cheung

Abstract

Recent work on non-autoregressive neural machine translation (NAT) that leverages alignment information to explicitly reduce the modality of target distribution has reported comparable performance with counterparts that tackle multi-modality problem by implicitly modeling dependencies. Effectiveness in handling alignment is vital for models that follow this approach, where a token reordering mechanism is typically involved and plays a vital role. We review the reordering capability of the respective mechanisms in recent NAT models, and our experimental results show that their performance is sub-optimal. We propose to learn a non-autoregressive language model (NALM) based on transformer which can be combined with Viterbi decoding to achieve better reordering performance. We evaluate the proposed NALM using the PTB dataset where sentences with words permuted in different ways are expected to have their ordering recovered. Our empirical results show that the proposed method can outperform the stateof-the-art reordering mechanisms under different word permutation settings, with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation cites Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
  • Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information cites Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
  • AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate cites Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information
  • AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate cites Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
  • AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate cites Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade
  • Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade cites Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information
  • Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade cites Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta PosteriorRaphael Shu, Jason Lee, Hideki Nakayama, Kyunghyun ChoAAAI 2020 · 125 citations
  2. Order-Agnostic Cross Entropy for Non-Autoregressive Machine TranslationCunxiao Du, Zhaopeng Tu, Jing JiangICML 2021 · 93 citations
  3. Fully Non-autoregressive Neural Machine Translation: Tricks of the TradeJiatao Gu, Xiang KongACL Findings 2021 · 84 citations
  4. Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering InformationQiu Ran, Yankai Lin, Peng Li, Jie ZhouAAAI 2021 · 82 citations
  5. AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and TranslateJongyoon Song, Sungwon Kim, Sungroh YoonEMNLP 2021 · 38 citations

Related papers

  1. Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine TranslationChenze Shao, Yang FengNeurIPS 2022 · 26 citations
  2. Aligned Cross Entropy for Non-Autoregressive Machine TranslationMarjan Ghazvininejad, Vladimir Karpukhin, Luke Zettlemoyer, Omer LevyICML 2020 · 121 citations
  3. Rephrasing the Reference for Non-autoregressive Machine TranslationChenze Shao, Jinchao Zhang, Jie Zhou, Yang FengAAAI 2023 · 6 citations
  4. MR-P: A Parallel Decoding Algorithm for Iterative Refinement Non-Autoregressive TranslationHao Cheng, Zhihua ZhangACL Findings 2022
  5. Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine TranslationChenze Shao, Jinchao Zhang, Yang Feng, Fandong Meng et al.AAAI 2020 · 95 citations
Dusk over the sea between two cliffs drawn in fine vertical lines