Information Flow Routes: Automatically Interpreting Language Models at Scale
Javier Ferrando, Elena Voita
Abstract
Information flows by routes inside the network via mechanisms implemented in the model. These routes can be represented as graphs where nodes correspond to token representations and edges to computations. We automatically build these graphs in a top-down manner, for each prediction leaving only the most important nodes and edges. In contrast to the existing workflows relying on activation patching, we do this through attribution: this allows us to efficiently uncover existing circuits with just a single forward pass. Unlike with patching, we do not need a human to carefully design prediction templates, and we can extract information flow routes for any prediction (not just the ones among the allowed templates). As a result, we can analyze model behavior in general, for specific types of predictions, or different domains. We experiment with Llama 2 and show that the role of some attention heads is overall important, e.g. previous token heads and subword merging heads. Next, we find similarities in Llama 2 behavior when handling tokens of the same part of speech. Finally, we show that some model components can be specialized on domains such as coding or multilingual texts. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
41 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Finding Transformer Circuits With Edge Pruning cites Towards Automated Circuit Discovery for Mechanistic Interpretability
- Finding Transformer Circuits With Edge Pruning cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
- Finding Transformer Circuits With Edge Pruning cites Language Models are Few-Shot Learners
- Finding Transformer Circuits With Edge Pruning cites How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
- Towards Automated Circuit Discovery for Mechanistic Interpretability cites Locating and Editing Factual Associations in GPT
- LLM Circuit Analyses Are Consistent Across Training and Scale cites Towards Automated Circuit Discovery for Mechanistic Interpretability
- LLM Circuit Analyses Are Consistent Across Training and Scale cites Locating and Editing Factual Associations in GPT
- LLM Circuit Analyses Are Consistent Across Training and Scale cites How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation cites Towards Automated Circuit Discovery for Mechanistic Interpretability
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation cites Language Models are Few-Shot Learners
- Locating and Editing Factual Associations in GPT cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
- Locating and Editing Factual Associations in GPT cites Language Models are Few-Shot Learners
- Mechanistic Detection and Mitigation of Hallucination in Large Reasoning Models cites How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
- How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model cites Locating and Editing Factual Associations in GPT
- How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
- How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model cites Language Models are Few-Shot Learners
- Knowledge Circuits in Pretrained Transformers cites Finding Transformer Circuits With Edge Pruning
- Knowledge Circuits in Pretrained Transformers cites Towards Automated Circuit Discovery for Mechanistic Interpretability
- Knowledge Circuits in Pretrained Transformers cites Locating and Editing Factual Associations in GPT
- Knowledge Circuits in Pretrained Transformers cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
Cited by top-tier papers41
- Finding Transformer Circuits With Edge PruningAdithya Bhaskar, Alexander Wettig, Dan Friedman, Danqi ChenNeurIPS 2024 · 72 citations
- LLM Circuit Analyses Are Consistent Across Training and ScaleCurt Tigges, Michael Hanna, Qinan Yu, Stella BidermanNeurIPS 2024 · 71 citations
- Knowledge Circuits in Pretrained TransformersYunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang et al.NeurIPS 2024 · 71 citations
- Mechanistic Detection and Mitigation of Hallucination in Large Reasoning ModelsZhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang et al.ICLR 2026 · 30 citations
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval AugmentationPatrick Kahardipraja, Reduan Achtibat, Thomas Wiegand, Wojciech Samek et al.NeurIPS 2025 · 13 citations
Builds on14
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Towards Automated Circuit Discovery for Mechanistic InterpretabilityArthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim et al.NeurIPS 2023 · 861 citations
- Investigating Gender Bias in Language Models Using Causal Mediation AnalysisJesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian et al.NeurIPS 2020 · 851 citations
- How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelMichael Hanna, Ollie Liu, Alexandre VariengienNeurIPS 2023 · 251 citations
Related papers
- Inserting Information Bottleneck for Attribution in TransformersZhiying Jiang, Raphael Tang, Ji Xin, Jimmy LinEMNLP Findings 2020 · 9 citations
- Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous ProcessHail Hochman, Natalie Shapira, Yoav GoldbergACL 2026
- Interpreting Context Look-ups in Transformers: Investigating Attention-MLP InteractionsClement Neo, Shay B. Cohen, Fazl BarezEMNLP 2024 · 3 citations
- FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language ModelsJintao Tong, Wenwei Jin, Pengda Qin, Anqi Li et al.NeurIPS 2025 · 31 citations
- Exploring the Translation Mechanism of Large Language ModelsHongbin Zhang, Kehai Chen, Xuefeng Bai, Xiucheng Li et al.NeurIPS 2025 · 4 citations
