EMNLP Findings2020
Pseudo-Bidirectional Decoding for Local Sequence Transduction
Wangchunshu Zhou, Tao Ge, Ke Xu
Abstract
Local sequence transduction (LST) tasks are sequence transduction tasks where there exists massive overlapping between the source and target sequences, such as Grammatical Error Correction (GEC) and spell or OCR correction. Previous work generally tackles LST tasks with standard sequence-to-sequence (seq2seq) models that generate output tokens from left to right and suffer from the issue of unbalanced outputs. Motivated by the characteristic of LST tasks, in this paper, we propose a simple but versatile approach named Pseudo-Bidirectional Decoding (PBD) for LST tasks. PBD copies the corresponding representation of source tokens to the decoder as pseudo future context to enable the decoder to attends to its bi-directional context. In addition, the bidirectional decoding scheme and the characteristic of LST tasks motivate us to share the encoder and the decoder of seq2seq models. The proposed PBD approach provides right side context information for the decoder and models the inductive bias of LST tasks, reducing the number of parameters by half and providing good regularization effects. Experimental results on several benchmark datasets show that our approach consistently improves the performance of standard seq2seq models on LST tasks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Related papers
- Copy That! Editing Sequences by Copying SpansSheena Panthaplackel, Miltiadis Allamanis, Marc BrockschmidtAAAI 2021 · 28 citations
- Reducing Sequence Length by Predicting Edit Spans with Large Language ModelsMasahiro Kaneko, Naoaki OkazakiEMNLP 2023 · 3 citations
- A Framework for Bidirectional Decoding: Case Study in Morphological InflectionMarc E. Canby, Julia HockenmaierEMNLP Findings 2023 · 4 citations
- Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language ModelsPeter West, Ximing Lu, Ari Holtzman, Chandra Bhagavatula et al.ACL 2021
- Improving Seq2Seq Grammatical Error Correction via Decoding InterventionsHouquan Zhou, Yumeng Liu, Zhenghua Li, Min Zhang et al.EMNLP Findings 2023 · 17 citations
