Eliminating the Impossible, Whatever Remains Must Be True: On Extracting and Applying Background Knowledge in the Context of Formal Explanations

The rise of AI methods to make predictions and decisions has led to a pressing need for more explainable artificial intelligence (XAI) methods. One common approach for XAI is to produce a post-hoc explanation, explaining why a black box ML model made a certain prediction. Formal approaches to post-hoc explanations provide succinct reasons for why a prediction was made, as well as why not another prediction was made. But these approaches assume that features are independent and uniformly distributed. While this means that "why" explanations are correct, they may be longer than required. It also means the "why not" explanations may be suspect as the counterexamples they rely on may not be meaningful. In this paper, we show how one can apply background knowledge to give more succinct "why" formal explanations, that are presumably easier to interpret by humans, and give more accurate "why not" explanations. In addition, we show how to use existing rule induction techniques to efficiently extract background information from a dataset, and also how to report which background information was used to make an explanation, allowing a human to examine it if they doubt the correctness of the explanation. Marital Status = Married? Education = Dropout? Relationship = Not-in-family? -0.2192 0.1063 -0.1561 -0.3850 yes no yes no yes no T2 (≥ 50k) Marital Status = Married? Occupation = Service? Hours/w > 45? -0.2231 0.0707 -0.0080 -0.2549 yes no yes no yes no T3 (≥ 50k) Relationship = Own-child? Education = Master? Education = Dropout? 0.1186 -0.3483 -0.2844 -0.0128 yes no yes no yes no (b) Boosted tree (Chen and Guestrin 2016) consisting of 3 trees with the depth of each tree at most 2.