Lune

EMNLP Findings2021

Sequence-to-Lattice Models for Fast Translation

Yuntian Deng, Alexander M. Rush

Abstract

Non-autoregressive machine translation (NAT) approaches enable fast generation by utilizing parallelizable generative processes. The remaining bottleneck in these models is their decoder layers; unfortunately unlike in autoregressive models (Kasai et al., 2020), removing decoder layers from NAT models significantly degrades accuracy. This work proposes a sequence-to-lattice model that replaces the decoder with a search lattice. Our approach first constructs a candidate lattice using efficient lookup operations, generates lattice scores from a deep encoder, and finally finds the best path using dynamic programming. Experiments on three machine translation datasets show that our method is faster than past non-autoregressive generation approaches, and more accurate than naively reducing the number of decoder layers.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Glancing Transformer for Non-Autoregressive Neural Machine Translation cites Cascaded Text Generation with Markov Transformers
  • Glancing Transformer for Non-Autoregressive Neural Machine Translation cites Non-Autoregressive Machine Translation with Latent Alignments
  • Cascaded Text Generation with Markov Transformers cites Non-Autoregressive Machine Translation with Latent Alignments
  • Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation cites Non-Autoregressive Machine Translation with Latent Alignments
  • Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade cites Glancing Transformer for Non-Autoregressive Neural Machine Translation
  • Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade cites Non-Autoregressive Machine Translation with Latent Alignments
  • Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade cites Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine TranslationJungo Kasai, Nikolaos Pappas, Hao Peng, James Cross et al.ICLR 2021 · 154 citations
  2. Fully Non-autoregressive Neural Machine Translation: Tricks of the TradeJiatao Gu, Xiang KongACL Findings 2021 · 84 citations
  3. Cascaded Text Generation with Markov TransformersYuntian Deng, Alexander M. RushNeurIPS 2020 · 16 citations
  4. Non-Autoregressive Machine Translation with Latent AlignmentsChitwan Saharia, William Chan, Saurabh Saxena, Mohammad NorouziEMNLP 2020 · 6 citations
  5. Glancing Transformer for Non-Autoregressive Neural Machine TranslationLihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang et al.ACL 2021

Related papers

  1. NAT4AT: Using Non-Autoregressive Translation Makes Autoregressive Translation Faster and BetterHuanran Zheng, Wei Zhu, Xiaoling WangWWW 2024 · 13 citations
  2. switch-GLAT: Multilingual Parallel Machine Translation Via Code-Switch DecoderZhenqiao Song, Hao Zhou, Lihua Qian, Jingjing Xu et al.ICLR 2022 · 13 citations
  3. Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering InformationQiu Ran, Yankai Lin, Peng Li, Jie ZhouAAAI 2021 · 82 citations
  4. Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine TranslationJunliang Guo, Xu Tan, Linli Xu, Tao Qin et al.AAAI 2020 · 91 citations
  5. RenewNAT: Renewing Potential Translation for Non-autoregressive TransformerPei Guo, Yisheng Xiao, Juntao Li, Min ZhangAAAI 2023 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines