ACL Findings2025
FADE: Why Bad Descriptions Happen to Good Features
Bruno Puri, Aakriti Jain, Elena Golimblevskaia, Patrick Kahardipraja, Thomas Wiegand, Wojciech Samek, Sebastian Lapuschkin
Abstract
Recent advances in mechanistic interpretability have highlighted the potential of automating interpretability pipelines in analyzing the latent representations within LLMs. While this may enhance our understanding of internal mechanisms, the field lacks standardized evaluation methods for assessing the validity of discovered features. We attempt to bridge this gap by introducing FADE: Feature Alignment to Description Evaluation, a scalable model-agnostic framework for automatically evaluating feature-to-description alignment. FADE evaluates alignment across four key metrics -Clarity, Responsiveness, Purity, and Faithfulness -and systematically quantifies the causes of the misalignment between features and their descriptions. We apply
FADE to analyze existing open-source feature descriptions and assess key components of automated interpretability pipelines, aiming to enhance the quality of descriptions. Our findings highlight fundamental challenges in generating feature descriptions, particularly for SAEs compared to MLP neurons, providing insights into the limitations and future directions of automated interpretability. We release FADE as an open-source package at: https://github.com/brunibrun/FADE.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Circuit Insights: Towards Interpretability Beyond Activations cites Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework
- Manipulating Feature Visualizations with Gradient Slingshots cites Labeling Neural Representations with Inverse Recognition
- Labeling Neural Representations with Inverse Recognition cites Manipulating Feature Visualizations with Gradient Slingshots
- Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language cites A Multimodal Automated Interpretability Agent
- Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language cites Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework
- Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework cites Labeling Neural Representations with Inverse Recognition
Cited by top-tier papers4
- Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description FrameworkLaura Kopf, Nils Feldhus, Kirill Bykov, Philine Lou Bommer et al.NeurIPS 2025 · 12 citations
- Manipulating Feature Visualizations with Gradient SlingshotsDilyara Bareeva, Marina M.-C. Höhne, Alexander Warnecke, Lukas Pirch et al.NeurIPS 2025 · 8 citations
- Circuit Insights: Towards Interpretability Beyond ActivationsElena Golimblevskaia, Aakriti Jain, Bruno Puri, Ammar Ibrahim et al.ICLR 2026 · 4 citations
- Semantic Regexes: Auto-Interpreting LLM Features with a Structured LanguageAngie W. Boggust, Donghao Ren, Yannick Assogba, Dominik Moritz et al.ICLR 2026 · 3 citations
Builds on11
- LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation DatasetLianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li et al.ICLR 2024 · 419 citations
- A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and ToxicityAndrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg et al.ICML 2024 · 177 citations
- Sparse Autoencoders Trained on the Same Data Learn Different FeaturesGonçalo Paulo, Nora BelroseICLR 2026 · 96 citations
- A Multimodal Automated Interpretability AgentTamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram et al.ICML 2024 · 57 citations
- Labeling Neural Representations with Inverse RecognitionKirill Bykov, Laura Kopf, Shinichi Nakajima, Marius Kloft et al.NeurIPS 2023 · 36 citations
Related papers
- Automatically Interpreting Millions of Features in Large Language ModelsGonçalo Paulo, Alex Mallen, Caden Juang, Nora BelroseICML 2025
- Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse AutoencodersXiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong et al.ACL 2026
- Enhancing Automated Interpretability with Output-Centric Feature DescriptionsYoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger et al.ACL 2025
- Revising and Falsifying Sparse Autoencoder Feature ExplanationsGeorge Ma, Samuel Pfrommer, Somayeh SojoudiNeurIPS 2025 · 7 citations
- Evaluating Neuron Explanations: A Unified Framework with Sanity ChecksTuomas P. Oikarinen, Ge Yan, Tsui-Wei WengICML 2025
