One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment
Hao Zong, Conghu Yuan, Chao Bei, Wentao Chen, Huan Liu, Kaiyu Huang, Degen Huang
Abstract
Current paradigms for empowering Large Language Models (LLMs) with multilingual capabilities rely heavily on massive instruction tuning. We challenge this view, proposing that the barrier is topological alignment, not data quantity. We introduce Hybrid Cross-Alignment (HCA) , fusing a frozen NLLB encoder with a Qwen decoder via a closed-loop dual-adapter architecture. HCA utilizes a Source-Side Adapter to precondition encoder features and a Query-Residual Adapter to preserve generative stability, bridged by an adaptive gated cross-modal interface. Our core finding is the phenomenon of “Source-Side Alignment Generalization.” We demonstrate that training HCA on a single language pair (German-English) unlocks state-of-the-art zero-shot transfer to dozens of unseen languages for X-to-English translation . Crucially, our “Oracle” experiments reveal that this single-pair training recovers over 96.7% of the performance achievable by training on all available pairs. This suggests that a highly generalizable, source-side projection protocol exists. Evaluated rigorously across COMET and chrF++ , our ∼ 5.25B-parameter model significantly outperforms larger baselines, surpassing TowerPlus-9B (+9.0 COMET on low-resource languages) and Aya-101 (13B). Furthermore, performance scales linearly with encoder size; upgrading from 600M to 1.3B yields immediate gains (+3.4 points on Gujarati) with minimal retraining cost.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on10
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- LLM Augmented LLMs: Expanding Capabilities through CompositionRachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta et al.ICLR 2024 · 51 citations
- TOWER+: Bridging Generality and Translation Specialization in Multilingual LLMsRicardo Rei, Nuno Miguel Guerreiro, José Pombal, João Alves et al.ACL 2026 · 34 citations
- Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained EncodersGuanhua Chen, Shuming Ma, Yun Chen, Li Dong et al.EMNLP 2021 · 30 citations
- Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-TrainingBo Zheng, Li Dong, Shaohan Huang, Saksham Singhal et al.EMNLP 2021 · 15 citations
Related papers
- Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible MultilingualityMengyu Bu, Yang FengACL 2026 · 2 citations
- Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMsDanni Liu, Jan NiehuesACL 2025 · 23 citations
- Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from TranslationJulian Spravil, Sebastian Houben, Sven BehnkeAAAI 2026
- LangBridge: Interpreting Image as a Combination of Language EmbeddingsJiaqi Liao, Yuwei Niu, Fanqing Meng, Hao Li et al.ICCV 2025
- X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at ScaleHaoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang et al.ICLR 2025
