ACL Findings2026
OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solving
Xinyu Zhang, Boxuan Zhang, Yuchen Wan, Lingling Zhang, Yixing Yao, Bifan Wei, Yaqiang Wu, Jun Liu
Abstract
While Large Language Models (LLMs) demonstrate remarkable reasoning, complex optimization tasks remain challenging, requiring domain knowledge and robust implementation. However, existing benchmarks focus narrowly on Mathematical Programming and Combinatorial Optimization, hindering comprehensive evaluation. To address this, we introduce Op-tiVerse, a comprehensive benchmark of 1,000 curated problems spanning neglected domains, including Stochastic Optimization, Dynamic Optimization, Game Optimization, and Optimal Control, across three difficulty levels: Easy, Medium, and Hard. The experiments with 22 LLMs of different sizes reveal sharp performance degradation on Hard problems, where even advanced models like GPT-5.2 and Gemini-3 struggle to exceed 27% accuracy. Through error analysis, we identify that modeling & logic errors remain the primary bottleneck. Consequently, we propose a Dual-View Auditor Agent that improves the accuracy of the LLM modeling process without introducing significant time overhead. OptiVerse will serve as a foundational platform for advancing LLMs in solving complex optimization challenges.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solving cites Chain-of-Experts: When LLMs Meet Complex Operations Research Problems
- Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solving cites OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
- Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solving cites Large Language Models as End-to-end Combinatorial Optimization Solvers
- OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models cites Chain-of-Experts: When LLMs Meet Complex Operations Research Problems
- Large Language Models as End-to-end Combinatorial Optimization Solvers cites Chain-of-Experts: When LLMs Meet Complex Operations Research Problems
- Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling cites Chain-of-Experts: When LLMs Meet Complex Operations Research Problems
- Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling cites OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Cited by top-tier papers1
Builds on12
- Chain-of-Experts: When LLMs Meet Complex Operations Research ProblemsZiyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu et al.ICLR 2024 · 136 citations
- OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language ModelsAli AhmadiTeshnizi, Wenzhi Gao, Madeleine UdellICML 2024 · 77 citations
- Solver-Informed RL: Grounding Large Language Models for Authentic Optimization ModelingYitian Chen, Jingfan Xia, Siyu Shao, Dongdong Ge et al.NeurIPS 2025 · 54 citations
- PhysReason: A Comprehensive Benchmark towards Physics-Based ReasoningXinyu Zhang, Yuxuan Dong, Yanrui Wu, Jiaxing Huang et al.ACL 2025 · 51 citations
- Large Language Models as End-to-end Combinatorial Optimization SolversXia Jiang, Yaoxin Wu, Minshuo Li, Zhiguang Cao et al.NeurIPS 2025 · 37 citations
Related papers
- Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side VerificationHaoyang Liu, Jie Wang, Boxuan Niu, Xiongwei Han et al.ICML 2026 · 4 citations
- OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search SpacesXiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding et al.ACL Findings 2026 · 2 citations
- OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity ScalingYitian Chen, Cheng Cheng, Yinan Sun, Zi Ling et al.ICML 2026 · 3 citations
- HeuriGym: An Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial OptimizationHongzheng Chen, Yingheng Wang, Yaohui Cai, Hins Hu et al.ICLR 2026 · 26 citations
- Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial OptimizationXia Jiang, Jing Chen, Cong Zhang, Jie Gao et al.ACL Findings 2026
