Lune

ACL Findings2025

How Programming Concepts and Neurons Are Shared in Code Language Models

Amir Hossein Kargaran, Yihong Liu, François Yvon, Hinrich Schütze

Abstract

Several studies have explored the mechanisms of large language models (LLMs) in coding tasks, but most have focused on programming languages (PLs) in a monolingual setting.In this paper, we investigate the relationship between multiple PLs and English in the concept space of LLMs.We perform a few-shot translation task on 21 PL pairs using two Llamabased models.By decoding the embeddings of intermediate layers during this task, we observe that the concept space is closer to English (including PL keywords) and assigns high probabilities to English tokens in the second half of the intermediate layers.We analyze neuron activations for 11 PLs and English, finding that while language-specific neurons are primarily concentrated in the bottom layers, those exclusive to each PL tend to appear in the top layers.For PLs that are highly aligned with multiple other PLs, identifying language-specific neurons is not feasible.These PLs also tend to have a larger keyword set than other PLs and are closer to the model's concept space regardless of the input/output PL in the translation task.Our findings provide insights into how LLMs internally represent PLs, revealing structural patterns in the model's concept space.Code is available at https://github.com/cisnlp/ code-specific-neurons.Java: String message = ""; -Python: message = ""

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • How do Large Language Models Handle Multilingualism? cites Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
  • Beyond Language Boundaries: Uncovering Programming Language Families for Code Language Models cites Unsupervised Translation of Programming Languages
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?Zhe Yin, Xiaodong Gu, Beijun ShenFSE 2026
  2. Beyond Language Boundaries: Uncovering Programming Language Families for Code Language ModelsShangbo Yun, Xiaodong Gu, Jianghong Huang, Beijun ShenFSE 2026

Builds on19

  1. Unsupervised Translation of Programming LanguagesBaptiste Rozière, Marie-Anne Lachaux, Lowik Chanussot, Guillaume LampleNeurIPS 2020 · 606 citations
  2. CodeGen: An Open Large Language Model for Code with Multi-Turn Program SynthesisErik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu et al.ICLR 2023 · 234 citations
  3. How do Large Language Models Handle Multilingualism?Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi et al.NeurIPS 2024 · 196 citations
  4. Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual TranslationBiao Zhang, Ankur Bapna, Rico Sennrich, Orhan FiratICLR 2021 · 97 citations
  5. Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary SpaceMor Geva, Avi Caciularu, Kevin Ro Wang, Yoav GoldbergEMNLP 2022 · 92 citations

Related papers

  1. The Emergence of Abstract Thought in Large Language Models Beyond Any LanguageYuxin Chen, Yiran Zhao, Yang Zhang, An Zhang et al.NeurIPS 2025 · 27 citations
  2. Cross-Lingual Generalization and Compression: From Language-Specific to Shared NeuronsFrederick Riemenschneider, Anette FrankACL 2025 · 3 citations
  3. The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language ModelJiawei Chen, Wentao Chen, Jing Su, Jingjing Xu et al.ICLR 2025
  4. Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual GeneralizationNingyu Xu, Qi Zhang, Jingting Ye, Menghan Zhang et al.EMNLP Findings 2023 · 3 citations
  5. Probing the Emergence of Cross-lingual Alignment during LLM TrainingHetong Wang, Pasquale Minervini, Edoardo M. PontiACL Findings 2024 · 13 citations
Dusk over the sea between two cliffs drawn in fine vertical lines