Lune

EMNLP Findings2023

A Framework for Bidirectional Decoding: Case Study in Morphological Inflection

Marc E. Canby, Julia Hockenmaier

Abstract

Transformer-based encoder-decoder models that generate outputs in a left-to-right fashion have become standard for sequence-tosequence tasks. In this paper, we propose a framework for decoding that produces sequences from the "outside-in": at each step, the model chooses to generate a token on the left, on the right, or join the left and right sequences. We argue that this is more principled than prior bidirectional decoders. Our proposal supports a variety of model architectures and includes several training methods, such as a dynamic programming algorithm that marginalizes out the latent ordering variable. Our model sets state-of-the-art (SOTA) on the 2022 and 2023 shared tasks, beating the next best systems by over 4.7 and 2.7 points in average accuracy respectively. The model performs particularly well on long sequences, can implicitly learn the split point of words composed of stem and affix, and performs better relative to the baseline on datasets that have fewer unique lemmas (but more examples per lemma). 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. Morphological Inflection: A Reality CheckJordan Kodner, Sarah R. B. Payne, Salam Khalifa, Zoey LiuACL 2023 · 6 citations
  2. One Source, Two Targets: Challenges and Rewards of Dual DecodingJitao Xu, François YvonEMNLP 2021

Related papers

  1. P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-TrainingJunwei Bao, Yifan Wang, Ying Jiangyong, Yeyun Gong et al.EMNLP Findings 2022
  2. Pseudo-Bidirectional Decoding for Local Sequence TransductionWangchunshu Zhou, Tao Ge, Ke XuEMNLP Findings 2020 · 4 citations
  3. Discovering Non-monotonic Autoregressive Orderings with Variational InferenceXuanlin Li, Brandon Trabucco, Dong Huk Park, Michael Luo et al.ICLR 2021 · 17 citations
  4. Learning and Analyzing Generation Order for Undirected Sequence ModelsYichen Jiang, Mohit BansalEMNLP Findings 2021 · 1 citation
  5. Compositional Generalization via Semantic TaggingHao Zheng, Mirella LapataEMNLP Findings 2021 · 22 citations
Dusk over the sea between two cliffs drawn in fine vertical lines