Extracting PAC Decision Trees from Black Box Binary Classifiers: The Gender Bias Study Case on BERT-based Language Models
Ana Ozaki, Roberto Confalonieri, Ricardo Guimarães, Anders Imenes
Abstract
Decision trees are a popular machine learning method, valued for their inherent explainability. In Explainable AI, decision trees serve as surrogate models for complex black box AI models or as approximations of parts of such models. A key challenge of this approach is assessing how accurately the extracted decision tree represents the original model and determining the extent to which it can be trusted as an approximation of its behaviour. In this work, we investigate the use of the Probably Approximately Correct (PAC) framework to provide a theoretical guarantee of fidelity for decision trees extracted from AI models. Leveraging the theoretical foundations of the PAC framework, we adapt a decision tree algorithm to ensure a PAC guarantee under specific conditions. We focus on binary classification and conduct experiments where we extract decision trees from BERT-based language models with PAC guarantees. Our results indicate occupational gender bias in these models, which confirm previous results in the literature. Additionally, the decision tree format enhances the visualization of which occupations are most impacted by social bias.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Related papers
- Safety Verification of Decision-Tree Policies in Continuous TimeChristian Schilling, Anna Lukina, Emir Demirovic, Kim Guldstrand LarsenNeurIPS 2023 · 5 citations
- Quantitative Verification of Neural Networks and Its Security ApplicationsTeodora Baluta, Shiqi Shen, Shweta Shinde, Kuldeep S. Meel et al.CCS 2019 · 115 citations
- D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic BiasBhavya Ghai, Klaus MuellerIEEE VIS 2022 · 45 citations
- Mind the Biases: Quantifying Cognitive Biases in Language Model PromptingRuixi Lin, Hwee Tou NgACL Findings 2023 · 10 citations
- Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language ModelsHannah Rose Kirk, Yennie Jun, Filippo Volpin, Haider Iqbal et al.NeurIPS 2021 · 243 citations
