Lune

ACL Findings2026

Beyond Neural Incompatibility: Cross-Scale Knowledge Transfer in Large Language Models through Latent Semantic Alignment

Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang

Abstract

Large Language Models (LLMs) encode substantial knowledge in their parameters, which can be located, traced, and analyzed. Despite recent progress in neural interpretability, it is still unclear how to transfer such knowledge in a fine-grained manner, namely parametric knowledge transfer (PKT). A central challenge is to make cross-scale transfer effective and efficient when source and target models differ in architecture and parameterization. Existing methods that directly reuse layer parameters are therefore strongly limited by neural incompatibility. In this paper, we identify latent semantic alignment as the key prerequisite for cross-scale knowledge transfer. Instead of directly moving layer parameters, our approach uses activations as the transfer medium. SE-MALIGN has two stages: an layer attribution stage that attributes task-relevant source layers and selects exactly one source layer for each target layer, and a semantic alignment stage that pairs them from shallow to deep and optimizes the target with source-side supervisory hidden states. The alignment is carried out in latent space. In the current realization, training follows a shallow-to-deep frontier schedule: at each stage, only the current target layer is trainable, the layer objective is a Fisher-weighted quadratic surrogate on target-space aligned logits, and the final output layer keeps KL distillation. The transferred object nonetheless remains the aligned representation itself. Evaluations on four benchmarks demonstrate the efficacy of our method. Further analysis reveals the key factors that ease cross-scale knowledge transfer and provides insights into the nature of latent semantic alignment.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  2. Isotropy in the Contextual Embedding Space: Clusters and ManifoldsXingyu Cai, Jiaji Huang, Yuchen Bian, Kenneth ChurchICLR 2021 · 50 citations
  3. Neuron-Level Knowledge Attribution in Large Language ModelsZeping Yu, Sophia AnaniadouEMNLP 2024 · 9 citations
  4. Physics of Language Models: Part 3.3, Knowledge Capacity Scaling LawsZeyuan Allen-Zhu, Yuanzhi LiICLR 2025 · 8 citations
  5. Information Flow Routes: Automatically Interpreting Language Models at ScaleJavier Ferrando, Elena VoitaEMNLP 2024 · 3 citations

Related papers

  1. Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language ModelsYuqiao Tan, Shizhu He, Kang Liu, Jun ZhaoACL 2025 · 2 citations
  2. Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric PerspectiveMing Zhong, Chenxin An, Weizhu Chen, Jiawei Han et al.ICLR 2024 · 16 citations
  3. MTA: Multi-Granular Trajectory Alignment for Large Language Model DistillationPham Khanh Chi, Quoc Phong Dao, Thuat Nguyen, Linh Ngo Van et al.ACL 2026
  4. Inheriting Generalizable Knowledge from LLMs to Diverse Vertical TasksChang Liu, boyu shi, Xu Yang, Qiufeng Wang et al.ICLR 2026
  5. Cross-Layer Distillation with Semantic CalibrationDefang Chen, Jian-Ping Mei, Yuan Zhang, Can Wang et al.AAAI 2021 · 368 citations
Dusk over the sea between two cliffs drawn in fine vertical lines