Lune

ICML2025Top-tier venue

Validating Mechanistic Interpretations: An Axiomatic Approach

Nils Palumbo, Ravi Mangal, Zifan Wang, Saranya Vijayakumar, Corina S. Pasareanu, Somesh Jha

Abstract

Mechanistic interpretability aims to reverse engineer the computation performed by a neural network in terms of its internal components. Although there is a growing body of research on mechanistic interpretation of neural networks, the notion of a mechanistic interpretation itself is often ad-hoc. Inspired by the notion of abstract interpretation from the program analysis literature that aims to develop approximate semantics for programs, we give a set of axioms that formally characterize a mechanistic interpretation as a description that approximately captures the semantics of the neural network under analysis in a compositional manner. We demonstrate the applicability of these axioms for validating mechanistic interpretations on an existing, wellknown interpretability study as well as on a new case study involving a Transformer-based model trained to solve the well-known 2-SAT problem.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Sparse Autoencoders Find Highly Interpretable Features in Language Models cites Towards Automated Circuit Discovery for Mechanistic Interpretability
  • Towards Automated Circuit Discovery for Mechanistic Interpretability cites Causal Abstractions of Neural Networks
  • Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees cites Towards Automated Circuit Discovery for Mechanistic Interpretability
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable GuaranteesItamar Hadad, Guy Katz, Shahaf BassanICLR 2026 · 10 citations

Builds on26

  1. Sparse Autoencoders Find Highly Interpretable Features in Language ModelsRobert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart et al.ICLR 2024 · 1,072 citations
  2. Towards Automated Circuit Discovery for Mechanistic InterpretabilityArthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim et al.NeurIPS 2023 · 861 citations
  3. Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li et al.NeurIPS 2023 · 728 citations
  4. Causal Abstractions of Neural NetworksAtticus Geiger, Hanson Lu, Thomas Icard, Christopher PottsNeurIPS 2021 · 516 citations
  5. On Completeness-aware Concept-Based Explanations in Deep Neural NetworksChih-Kuan Yeh, Been Kim, Sercan Ömer Arik, Chun-Liang Li et al.NeurIPS 2020 · 390 citations

Related papers

  1. Tracking Equivalent Mechanistic Interpretations Across Neural NetworksAlan Sun, Mariya TonevaICLR 2026 · 1 citation
  2. A Toy Model of Universality: Reverse Engineering how Networks Learn Group OperationsBilal Chughtai, Lawrence Chan, Neel NandaICML 2023 · 144 citations
  3. Compact Proofs of Model Performance via Mechanistic InterpretabilityJason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong et al.NeurIPS 2024 · 20 citations
  4. DePass: Unified Feature Attributing by Simple Decomposed Forward PassXiangyu Hong, Che Jiang, Kai Tian, Biqing Qi et al.NeurIPS 2025 · 4 citations
  5. Towards a Unified and Verified Understanding of Group-Operation NetworksWilson Wu, Louis Jaburi, Jacob Drori, Jason GrossICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines