Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang, Libo Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, Wanxiang Che
Abstract
Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Currently, PoT primarily uses Python. However, relying solely on a single language may result in suboptimal solutions and overlook the potential benefits of other programming languages. In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models. The effectiveness of each language varies depending on the specific scenarios. Inspired by this, we propose a task and model agnostic approach called MultiPoT, which harnesses strength and diversity from various languages. Experimental results reveal that it significantly outperforms Python Self-Consistency. Furthermore, it achieves comparable or superior performance compared to the best monolingual PoT in almost all tasks across all models. In particular, MultiPoT achieves more than 4.6% improvement on average on ChatGPT (gpt-3.5-turbo-0701) 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- PAL: Program-aided Language Models cites Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- ViperGPT: Visual Inference via Python Execution for Reasoning cites PAL: Program-aided Language Models
- Automated Creation of Reusable and Diverse Toolsets for Enhancing LLM Reasoning cites PAL: Program-aided Language Models
- Executable Code Actions Elicit Better LLM Agents cites PAL: Program-aided Language Models
- Executable Code Actions Elicit Better LLM Agents cites ViperGPT: Visual Inference via Python Execution for Reasoning
Cited by top-tier papers2
- ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code GenerationXuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen et al.ACL 2025 · 62 citations
- Automated Creation of Reusable and Diverse Toolsets for Enhancing LLM ReasoningZhiyuan Ma, Zhenya Huang, Jiayu Liu, Minmao Wang et al.AAAI 2025 · 8 citations
Builds on11
- ViperGPT: Visual Inference via Python Execution for ReasoningDídac Surís, Sachit Menon, Carl VondrickICCV 2023 · 732 citations
- PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon et al.ICML 2023 · 700 citations
- Executable Code Actions Elicit Better LLM AgentsXingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang et al.ICML 2024 · 436 citations
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve ThemMirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann et al.ACL Findings 2023 · 253 citations
- CodeGen: An Open Large Language Model for Code with Multi-Turn Program SynthesisErik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu et al.ICLR 2023 · 234 citations
Related papers
- MultiLingPoT: Boosting Mathematical Reasoning in LLMs through Multilingual Program IntegrationNianqi Li, Zujie Liang, Siyu Yuan, Jiaqing Liang et al.EMNLP Findings 2025
- Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual EnvironmentsPatomporn Payoungkhamdee, Pume Tuchinda, Jinheon Baek, Samuel Cahyawijaya et al.ACL Findings 2025
- Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM ReasoningRaman Saparkhan, Majd Hawasly, Md. Rizwan Parvez, Mohammad RazaACL Findings 2026
- How Do Humans Write Code? Large Models Do It the Same Way TooLong Li, Xuzheng He, Haozhe Wang, Linlin Wang et al.EMNLP 2024 · 1 citation
- Dynamic Planning for LLM-based Graphical User Interface AutomationShaoqing Zhang, Zhuosheng Zhang, Kehai Chen, Xinbei Ma et al.EMNLP Findings 2024 · 10 citations
