Understanding Large Language Model Behaviors Through Interactive Counterfactual Generation and Analysis

A 23-year-old pregnant woman at 22 weeks gestation presents burning upon urination... Which of the following is the best treatment for this patient?

The answer is D. Nitrofurantoin CONTAIN "Nitrofurantoin" Outcome: A A 23-year-old pregnant woman at 22 weeks gestation presents burning upon urination... Which of the following is the best treatment for this patient? The answer is B. Ceftriaxone The Original Prompt and the LLM Response A Counterfactual Prompt and the LLM Response

Fig. 1: LLM Analyzer enables users to analyze and understand LLM behaviors through meaningful counterfactuals. From the user's input prototype sentences, such as a medical question in this example, the system generates meaningful segments for perturbation. (A) Users can interactively adjust their granularities and specify alternative segments for replacements, which are then used by the system to create meaningful counterfactuals. The system enables users to analyze the LLM by inspecting and interactively aggregating the counterfactual examples in a table-based visualization. (B) The table header shows the segments' text, dependencies, and feature attributions. (C) By grouping the counterfactuals by segments of interest, users can assess their joint influence on the predictions. (D) In addition to the statistical results, users could view concrete examples to understand the LLM's prediction in alternative scenarios.