Validating Mechanistic Interpretations: An Axiomatic Approach
Nils Palumbo, Ravi Mangal, Zifan Wang, Saranya Vijayakumar, Corina S. Pasareanu, Somesh Jha
Abstract
Mechanistic interpretability aims to reverse engineer the computation performed by a neural network in terms of its internal components. Although there is a growing body of research on mechanistic interpretation of neural networks, the notion of a mechanistic interpretation itself is often ad-hoc. Inspired by the notion of abstract interpretation from the program analysis literature that aims to develop approximate semantics for programs, we give a set of axioms that formally characterize a mechanistic interpretation as a description that approximately captures the semantics of the neural network under analysis in a compositional manner. We demonstrate the applicability of these axioms for validating mechanistic interpretations on an existing, wellknown interpretability study as well as on a new case study involving a Transformer-based model trained to solve the well-known 2-SAT problem.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Sparse Autoencoders Find Highly Interpretable Features in Language Models cites Towards Automated Circuit Discovery for Mechanistic Interpretability
- Towards Automated Circuit Discovery for Mechanistic Interpretability cites Causal Abstractions of Neural Networks
- Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees cites Towards Automated Circuit Discovery for Mechanistic Interpretability
Cited by top-tier papers1
Builds on26
- Sparse Autoencoders Find Highly Interpretable Features in Language ModelsRobert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart et al.ICLR 2024 · 1,072 citations
- Towards Automated Circuit Discovery for Mechanistic InterpretabilityArthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim et al.NeurIPS 2023 · 861 citations
- Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li et al.NeurIPS 2023 · 728 citations
- Causal Abstractions of Neural NetworksAtticus Geiger, Hanson Lu, Thomas Icard, Christopher PottsNeurIPS 2021 · 516 citations
- On Completeness-aware Concept-Based Explanations in Deep Neural NetworksChih-Kuan Yeh, Been Kim, Sercan Ömer Arik, Chun-Liang Li et al.NeurIPS 2020 · 390 citations
Related papers
- Tracking Equivalent Mechanistic Interpretations Across Neural NetworksAlan Sun, Mariya TonevaICLR 2026 · 1 citation
- A Toy Model of Universality: Reverse Engineering how Networks Learn Group OperationsBilal Chughtai, Lawrence Chan, Neel NandaICML 2023 · 144 citations
- Compact Proofs of Model Performance via Mechanistic InterpretabilityJason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong et al.NeurIPS 2024 · 20 citations
- DePass: Unified Feature Attributing by Simple Decomposed Forward PassXiangyu Hong, Che Jiang, Kai Tian, Biqing Qi et al.NeurIPS 2025 · 4 citations
- Towards a Unified and Verified Understanding of Group-Operation NetworksWilson Wu, Louis Jaburi, Jacob Drori, Jason GrossICLR 2025
