DARE: Disentanglement-Augmented Rationale Extraction

Rationale extraction can be considered as a straightforward method of improving the model explainability, where rationales are a subsequence of the original inputs, and can be extracted to support the prediction results. Existing methods are mainly cascaded with the selector which extracts the rationale tokens, and the predictor which makes the prediction based on selected tokens. Since previous works fail to fully exploit the original input, where the information of non-selected tokens is ignored, in this paper, we propose a Disentanglement-Augmented Rationale Extraction (DARE) method, which encapsulates more information from the input to extract rationales. Specifically, it first disentangles the input into the rationale representations and the non-rationale ones, and then learns more comprehensive rationale representations for extracting by minimizing the mutual information (MI) between the two disentangled representations. Besides, to improve the performance of MI minimization, we develop a new MI estimator by exploring existing MI estimation methods. Extensive experimental results on three real-world datasets and simulation studies clearly validate the effectiveness of our proposed method. Code is released at https://github.com/yuelinan/DARE . ⇤ Corresponding Author 36th Conference on Neural Information Processing Systems (NeurIPS 2022). whole text as the input and generate the accurate but uninterpretable representations as most DNNs do to predict the result. Then, the guider utilizes the above representations to guide the predictor to yield more comprehensive task-related representations with an adversarial-based method. Since this method fails to utilize the information of the original text, where the non-rationale tokens are ignored, we argue that this "guidance pattern" can be further explored to improve the rationale extraction. After hearing, our court identified that the defendant and the victim had a dispute caused by trivial. The defendant slashed the victim with a knife, which caused a serious injury to the victim. Soon after, the defendant was under arrest by policeman...... selector predictor guider Charge: Crime of intentional injury After hearing, our court identified that the defendant and the victim had a dispute caused by trivial. The defendant slashed the victim with a knife, which caused a serious injury to the victim. Soon after, the defendant was under arrest by policeman...... After hearing, our court identified that the defendant and the victim had a dispute caused by trivial. The defendant slashed the victim with a knife, which caused a serious injury to the victim. Soon after, the defendant was under arrest by policeman......