CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Models

Counterfactual reasoning is widely recognized as one of the most challenging and intricate aspects of causality in artificial intelligence. In this paper, we evaluate the performance of large language models (LLMs) in counterfactual reasoning. In contrast to previous studies that primarily focus on commonsense causal reasoning, where LLMs often rely on prior knowledge for inference, we specifically assess their ability to perform counterfactual inference using a set of formal rules. To support this evaluation, we introduce a new benchmark dataset, Coun-terBench, comprising 1.2K counterfactual reasoning questions. The dataset is designed with varying levels of difficulty, diverse causal graph structures, distinct types of counterfactual questions, and multiple nonsensical name variants. Our experiments demonstrate that counterfactual reasoning poses a significant challenge for LLMs, with most models performing at levels comparable to random guessing. To enhance LLM's counterfactual reasoning ability, we propose a novel reasoning paradigm, CoIn, which guides LLMs through iterative reasoning and backtracking to systematically explore counterfactual solutions. Experimental results show that our method significantly improves LLM performance on counterfactual reasoning tasks and consistently enhances performance across different LLMs. Our dataset is available at https://huggingface.co/datasets/CounterBench /CounterBench . Type Query Template Example Causal graph Basic We know that X causes V1, V1 causes V2, V2 causes V3, and V3 causes V4, V4 causes V5, V5 causes Y. Would Y occur if not X instead of X? Joint We know that X causes V1, V1 causes V2, V2 and V1 together cause V3, V3 causes V4, V4 and X together cause V5, and V5 causes Y. Would Y occur if not X and not V3? Nested We know that X causes V1, V1 causes V2, V2 and V1 together cause V3, V3 causes V4, V4 and V2 together cause V5, and V5 causes Y. Assume not X, and based on this assumption, further suppose not V4. Would Y occur? Conditional We know that X and V1 together cause V2, V2 causes V3, V3 causes V4, V4 causes V5, V5 causes Y. We observed V1. Would Y occur if not X instead of X? Backdoor We know that V1 causes X, X and V1 together cause V2, V2 causes V3, V3 and X together cause V4, V4 causes V5, and V5 causes Y.