Lune

ACL Findings2025

Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments

Patomporn Payoungkhamdee, Pume Tuchinda, Jinheon Baek, Samuel Cahyawijaya, Can Udomcharoenchaikit, Potsawee Manakul, Peerat Limkonchotiwat, Ekapol Chuangsuwanich, Sarana Nutanong

Abstract

Multi-step reasoning is essential for large language models (LLMs), yet multilingual performance remains challenging. While Chain-of-Thought (CoT) prompting improves reasoning, it struggles with non-English languages due to the entanglement of reasoning and execution. Program-of-Thought (PoT) prompting separates reasoning from execution, offering a promising alternative but shifting the challenge to generating programs from non-English questions. We propose a framework to evaluate PoT by separating multilingual reasoning from code execution to examine (i) the impact of fine-tuning on question-reasoning alignment and (ii) how reasoning quality affects answer correctness. Our findings demonstrate that PoT fine-tuning substantially enhances multilingual reasoning, outperforming CoT fine-tuned models. We further demonstrate a strong correlation between reasoning quality (measured through code quality) and answer accuracy, highlighting its potential as a test-time performance improvement heuristic. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites PAL: Program-aided Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • PAL: Program-aided Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • PAL: Program-aided Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  3. PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon et al.ICML 2023 · 700 citations
  4. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  5. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsLonghui Yu, Weisen Jiang, Han Shi, Jincheng Yu et al.ICLR 2024 · 637 citations

Related papers

  1. MultiLingPoT: Boosting Mathematical Reasoning in LLMs through Multilingual Program IntegrationNianqi Li, Zujie Liang, Siyu Yuan, Jiaqing Liang et al.EMNLP Findings 2025
  2. Empowering Multi-step Reasoning across Languages via Program-Aided Language ModelsLeonardo Ranaldi, Giulia Pucci, Barry Haddow, Alexandra BirchEMNLP 2024 · 2 citations
  3. Self-Polish: Enhance Reasoning in Large Language Models via Problem RefinementZhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng et al.EMNLP Findings 2023 · 16 citations
  4. Python is Not Always the Best Choice: Embracing Multilingual Program of ThoughtsXianzhen Luo, Qingfu Zhu, Zhiming Zhang, Libo Qin et al.EMNLP 2024 · 1 citation
  5. Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What MattersBoshi Wang, Sewon Min, Xiang Deng, Jiaming Shen et al.ACL 2023 · 100 citations
Dusk over the sea between two cliffs drawn in fine vertical lines