Lune

EMNLP Findings2021

Japanese Zero Anaphora Resolution Can Benefit from Parallel Texts Through Neural Transfer Learning

Masato Umakoshi, Yugo Murawaki, Sadao Kurohashi

Abstract

Parallel texts of Japanese and a non-pro-drop language have the potential of improving the performance of Japanese zero anaphora resolution (ZAR) because pronouns dropped in the former are usually mentioned explicitly in the latter. However, rule-based cross-lingual transfer is hampered by error propagation in an NLP pipeline and the frequent lack of transparency in translation correspondences. In this paper, we propose implicit transfer by injecting machine translation (MT) as an intermediate task between pretraining and ZAR. We employ a pretrained BERT model to initialize the encoder part of the encoder-decoder model for MT, and eject the encoder part for finetuning on ZAR. The proposed framework empirically demonstrates that ZAR performance can be improved by transfer learning from MT. In addition, we find that the incorporation of the masked language model training into MT leads to further gains.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Exploring and Predicting Transferability across NLP Tasks cites Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  2. Incorporating BERT into Neural Machine TranslationJinhua Zhu, Yingce Xia, Lijun Wu, Di He et al.ICLR 2020 · 391 citations
  3. Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut et al.ACL 2020 · 168 citations
  4. Exploring and Predicting Transferability across NLP TasksTu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni et al.EMNLP 2020 · 104 citations
  5. Pseudo Zero Pronoun Resolution Improves Zero Anaphora ResolutionRyuto Konno, Shun Kiyono, Yuichiroh Matsubayashi, Hiroki Ouchi et al.EMNLP 2021 · 6 citations

Related papers

  1. Exploring Cross-Lingual Transfer Learning with Unsupervised Machine TranslationChao Wang, Judith Gaspers, Quynh Ngoc Thi Do, Hui JiangACL Findings 2021 · 3 citations
  2. Acquiring Knowledge from Pre-Trained Model to Neural Machine TranslationRongxiang Weng, Heng Yu, Shujian Huang, Shanbo Cheng et al.AAAI 2020 · 71 citations
  3. Multilingual Transfer Learning for QA using Translation as Data AugmentationMihaela A. Bornea, Lin Pan, Sara Rosenthal, Radu Florian et al.AAAI 2021 · 45 citations
  4. Coupling Context Modeling with Zero Pronoun Recovering for Document-Level Natural Language GenerationXin Tan, Longyin Zhang, Guodong ZhouEMNLP 2021 · 6 citations
  5. MMAR: Multilingual and Multimodal Anaphora Resolution in Instructional VideosCennet Oguz, Pascal Denis, Simon Ostermann, Emmanuel Vincent et al.EMNLP Findings 2024
Dusk over the sea between two cliffs drawn in fine vertical lines