EMNLP Findings2025
MultiLingPoT: Boosting Mathematical Reasoning in LLMs through Multilingual Program Integration
Nianqi Li, Zujie Liang, Siyu Yuan, Jiaqing Liang, Feng Wei, Yanghua Xiao
Abstract
Program-of-Thought, which aims to use program instead of natural language in reasoning, is an important way for LLMs to solve mathematical problems. Since different programming languages excel in different areas, it is natural to use the most suitable language for solving specific problems. However, current research only focuses on single language PoT, ignoring the differences between programming languages. Therefore, this paper proposes a multilingual programme reasoning method, MultiLingPoT, and deeply explores the impact of multilingual integration in the training and inference. This method allows the model to answer questions using multiple languages by fine-tuning on multilingual data and improving individual language's reasoning accuracy by 2.5%. Additionally, prior and posterior selection methods are used to help the model select the most suitable language during inference, and achieves 8% performance gains. Finally, our code metric analysis shows that language differences manifest in encapsulation levels and implementation granularity, while strategic deviation from language conventions can enhances code performance. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on19
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesZhangyin Feng, Daya Guo, Duyu Tang, Nan Duan et al.EMNLP Findings 2020 · 2,828 citations
- ZeRO: memory optimizations toward training trillion parameter modelsSamyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong HeSC 2020 · 852 citations
- ViperGPT: Visual Inference via Python Execution for ReasoningDídac Surís, Sachit Menon, Carl VondrickICCV 2023 · 732 citations
- PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon et al.ICML 2023 · 700 citations
Related papers
- Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual EnvironmentsPatomporn Payoungkhamdee, Pume Tuchinda, Jinheon Baek, Samuel Cahyawijaya et al.ACL Findings 2025
- Python is Not Always the Best Choice: Embracing Multilingual Program of ThoughtsXianzhen Luo, Qingfu Zhu, Zhiming Zhang, Libo Qin et al.EMNLP 2024 · 1 citation
- How Do Humans Write Code? Large Models Do It the Same Way TooLong Li, Xuzheng He, Haozhe Wang, Linlin Wang et al.EMNLP 2024 · 1 citation
- Once Upon an Input: Reasoning via Per-Instance Program SynthesisAdam Stein, Neelay Velingker, Mayur Naik, Eric WongNeurIPS 2025
- Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-AttentionNegar Foroutan, Mohammadreza Banaei, Karl Aberer, Antoine BosselutEMNLP Findings 2023 · 2 citations
