EvidenceNet: Evidence Fusion Network for Fact Verification

Fact verification is the task to verify a given 001 claim according to extracted evidence sen-002 tences. Most existing works use whole ev-003 idence sentences or break them into phrases 004 to perform evidence interaction, where evi-005 dence is treated either too coarsely or over 006 fragmented. We also find that many models 007 suffer from exposure bias, which finally leads 008 to them only paying attention to the evidence 009 ranked higher by previous steps while failing 010 to recognize crucial pieces from all candidates. 011 In this paper, we propose an Evidence Decom-012 position Graph Network (EDGN), which de-013 composes each evidence sentence, especially 014 the complex ones, into several simple sen-015 tences, highlighting the required key infor-016 mation without losing sentence structure and 017 meaning. EDGN also absorbs a simple but ef-018 fective evidence shuffling method to mitigate 019 exposure bias. Experiments on the FEVER 020 benchmark show our model can take all ev-021 idence candidates into account, distill neces-022 sary key information from complex evidence, 023 and outperform existing methods in the litera-024 ture. We will release our code to the commu-025 nity for further exploration. 026 1 Introduction 027 FEVER (Thorne et al., 2018) is a Fact Extraction 028 and Verification task, where a system is asked to 029 predict whether a given claim is supported, re-030 futed, or can not be verified based on a Wikipedia 031 dump. Most existing works follow a three-step 032 pipeline (Thorne et al., 2018): (i) Retrieve rele-033 vant pages from Wikipedia dump. (ii) Extract evi-034 dence sentences from retrieved pages. (iii) Verify 035 the given claim based on the extracted evidence 036 candidates. Hanselowski et al. (2018) and Liu 037 et al. (2020) have contributed efficient and effec-038 tive methods for the retrieval and extraction steps 039 respectively. Table 1 shows an example with ex-040 tracted evidence candidates present in a descending 041 tion model. However, this may make the verifica-084 tion model learn shortcuts, e.g., only concentrating 085 on evidence pieces close to the claim in the input 086 sequences, but probably failing to recognize the 087 gold evidence from imperfect evidence extractions. 088 This exposure bias will limit the model's general-089 ization ability at real-world scenarios, where the 090 positions of the gold evidence are relatively scat-091 tered. We think that one way to prevent learning 092 such shortcuts may be to shuffle the candidates 093 during training, and push the model to learn to 094 recognize crucial evidence from tough cases, thus 095 improve its generalization ability. 096 In this paper, we propose an Evidence Decom-097 position Graph Network (EDGN), a sub-sentence 098 level graph network with the evidence decompo-099 sition and reordering mechanism. Specifically, 100 EDGN decomposes each evidence candidate into 101 several clause-level sub-evidence pieces, highlight-102 ing key information in evidence candidates without 103 losing much syntactic and semantic information 104 in the original sentences. We also connect sub-105 evidence pieces from the same evidence to form 106 an sub-evidence interaction graph. With a random 107 shuffle mechanism, we reorder the evidence can-108 didates to help EDGN learn to recognize crucial 109 evidence. Then, we apply a multi-layer Graph At-110 tention Network (GAT) to the sub-evidence interac-111 ponent. (1) w/ Dep-based Decomposition. We use 351 simple rule-based evidence decomposition method 352 with the "conjunction" relationship in dependency 353 parsing instead of LaserTagger (elaborated in Ap-354 pendix C). (2) w/o Evidence Decomposition. We 355 use the original retrieved evidence without splitting 356 them into sub-evidence pieces. (3) w/o Graph In-357 teraction. We remove the graph interaction module, 358 and directly use the outcomes of PLM for veracity 359 prediction and gold evidence prediction. (4) w/o 360 Auxiliary Task. We remove the auxiliary task, gold 361 evidence prediction, and optimize the model solely 362 with the veracity prediction target. 363 In Table 3, without each component, the per-364 formance on the validation set drops consistently, 365 demonstrating the effectiveness of our proposed 366 modules. Particularly, without evidence decompo-367 sition, our model performs worse on the validation 368 set by 0.79% and 0.78% in accuracy and FEVER 369 score. Organizing the evidence with similar granu-370 larity with the given claim, our model could capture 371 the relation between evidence and the claim more 372 precisely. With a rule-based evidence splitter, the 373 performances slightly drop, but are still better than 374 the setting without evidence decomposition. Even 375 rule-based evidence decomposition works for our 376 proposed method, and more sophisticated evidence 377 splitter brings further improvements. Without the 378 auxiliary gold evidence prediction task, both the 379 l