Lune

ACL Findings2022

Interpretable Research Replication Prediction via Variational Contextual Consistency Sentence Masking

Tianyi Luo, Rui Meng, Xin Wang, Yang Liu

Abstract

Research Replication Prediction (RRP) is the task of predicting whether a published research result can be replicated or not. Building an interpretable neural text classifier for RRP promotes the understanding of why a research paper is predicted as replicable or non-replicable and therefore makes its real-world application more reliable and trustworthy. However, the prior works on model interpretation mainly focused on improving the model interpretability at the word/phrase level, which are insufficient especially for long research papers in RRP. Furthermore, the existing methods cannot utilize a large size of unlabeled dataset to further improve the model interpretability. To address these limitations, we aim to build an interpretable neural model which can provide sentence-level explanations and apply weakly supervised approach to further leverage the large corpus of unlabeled datasets to boost the interpretability in addition to improving prediction performance as existing works have done. In this work, we propose the Variational Contextual Consistency Sentence Masking (VCCSM) method to automatically extract key sentences based on the context in the classifier, using both labeled and unlabeled datasets. Results of our experiments on RRP along with European Convention of Human Rights (ECHR) datasets demonstrate that VCCSM is able to improve the model interpretability for the long document classification tasks using the area over the perturbation curve and post-hoc accuracy as evaluation metrics.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification cites Unsupervised Data Augmentation for Consistency Training
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Unsupervised Data Augmentation for Consistency TrainingQizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong et al.NeurIPS 2020 · 2,774 citations
  2. MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text ClassificationJiaao Chen, Zichao Yang, Diyi YangACL 2020 · 340 citations
  3. Explaining Black Box Predictions and Unveiling Data Artifacts through Influence FunctionsXiaochuang Han, Byron C. Wallace, Yulia TsvetkovACL 2020 · 91 citations
  4. Learning Variational Word Masks to Improve the Interpretability of Neural Text ClassifiersHanjie Chen, Yangfeng JiEMNLP 2020 · 45 citations
  5. Research Replication Prediction Using Weakly Supervised LearningTianyi Luo, Xingyu Li, Hainan Wang, Yang LiuEMNLP Findings 2020 · 8 citations

Related papers

  1. SELFEXPLAIN: A Self-Explaining Architecture for Neural Text ClassifiersDheeraj Rajagopal, Vidhisha Balachandran, Eduard H. Hovy, Yulia TsvetkovEMNLP 2021 · 39 citations
  2. QUASER: Question Answering with Scalable Extractive RationalizationAsish Ghoshal, Srinivasan Iyer, Bhargavi Paranjape, Kushal Lakhotia et al.SIGIR 2022 · 2 citations
  3. Generating Hierarchical Explanations on Text Classification via Feature Interaction DetectionHanjie Chen, Guangtao Zheng, Yangfeng JiACL 2020 · 85 citations
  4. Explanation Consistency Training: Facilitating Consistency-Based Semi-Supervised Learning with InterpretabilityTao Han, Wei-Wei Tu, Yufeng LiAAAI 2021 · 25 citations
  5. Mitigating the Inconsistency Between Word Saliency and Model Confidence with Pathological Contrastive TrainingPengwei Zhan, Yang Wu, Shaolei Zhou, Yunjian Zhang et al.ACL Findings 2022 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines