Lune

EMNLP Findings2021

Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing

Akshat Shrivastava, Pierce Chuang, Arun Babu, Shrey Desai, Abhinav Arora, Alexander Zotov, Ahmed Aly

Abstract

An effective recipe for building seq2seq, nonautoregressive, task-oriented parsers to map utterances to semantic frames proceeds in three steps: encoding an utterance x, predicting a frame's length |y|, and decoding a |y|sized frame with utterance and ontology tokens. Though empirically strong, these models are typically bottlenecked by length prediction, as even small inaccuracies change the syntactic and semantic characteristics of resulting frames. In our work, we propose span pointer networks, non-autoregressive parsers which shift the decoding task from text generation to span prediction; that is, when imputing utterance spans into frame slots, our model produces endpoints (e.g., [i, j]) as opposed to text (e.g., "6pm"). This natural quantization of the output space also provides consistency in the length prediction task, allowing our length predictor to be responsible for frame syntax and the decoder for frame syntax, creating a coarse-to-fine model. We evaluate our approach on several task-oriented semantic parsing datasets. Notably, we bridge the quality gap between non-autogressive and autoregressive parsers, achieving 87 EM on TOPv2 (Chen et al., 2020). Furthermore, due to our more consistent gold frames, we show strong improvements in model generalization in both cross-domain and cross-lingual transfer in low-resource settings. Finally, due to our diminished output vocabulary, we observe 70% reduction in latency and 83% in memory at beam size 5 compared to prior nonautoregressive parsers.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Non-autoregressive Machine Translation with Disentangled Context Transformer cites Understanding Knowledge Distillation in Non-autoregressive Machine Translation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-Based EncoderElman Mansimov, Yi ZhangAAAI 2022 · 16 citations
  2. Towards Zero-Shot Multilingual Transfer for Code-Switched ResponsesTing-Wei Wu, Changsheng Zhao, Ernie Chang, Yangyang Shi et al.ACL 2023 · 2 citations

Builds on8

  1. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  2. Large Batch Optimization for Deep Learning: Training BERT in 76 minutesYang You, Jing Li, Sashank J. Reddi, Jonathan Hseu et al.ICLR 2020 · 1,170 citations
  3. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  4. Understanding Knowledge Distillation in Non-autoregressive Machine TranslationChunting Zhou, Jiatao Gu, Graham NeubigICLR 2020 · 235 citations
  5. Non-autoregressive Machine Translation with Disentangled Context TransformerJungo Kasai, James Cross, Marjan Ghazvininejad, Jiatao GuICML 2020 · 113 citations

Related papers

  1. Non-Autoregressive Sentence OrderingYi Bin, Wenhao Shi, Bin Ji, Jipeng Zhang et al.EMNLP Findings 2023 · 2 citations
  2. Diagnosing Transformers in Task-Oriented Semantic ParsingShrey Desai, Ahmed AlyACL Findings 2021 · 5 citations
  3. LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic ParsingDora Jambor, Dzmitry BahdanauACL 2022
  4. Span-based Semantic Parsing for Compositional GeneralizationJonathan Herzig, Jonathan BerantACL 2021
  5. Joint Multi-Decoder Framework with Hierarchical Pointer Network for Frame Semantic ParsingXudong Chen, Ce Zheng, Baobao ChangACL Findings 2021 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines