StepFun-Formalizer: Unlocking the Autoformalization Potential of LLMs Through Knowledge-Reasoning Fusion
Yutong Wu, Di Huang, Ruosi Wan, Yue Peng, Shijie Shang, Chenrui Cao, Lei Qi, Rui Zhang, Xishan Zhang, Zidong Du, Jie Yang, Xing Hu
Abstract
Autoformalization aims to translate natural-language mathematical statements into a formal language. While LLMs have accelerated progress in this area, existing methods still suffer from low accuracy. We identify two key abilities for effective autoformalization: comprehensive mastery of formal-language domain knowledge, and reasoning capability of natural language problem understanding and informal-formal alignment. Without the former, a model cannot identify the correct formal objects; without the latter, it struggles to interpret real-world contexts and map them precisely into formal expressions. To address these gaps, we introduce ThinkingF, a data synthesis and training pipeline that improves both abilities. First, we construct two datasets: one by distilling and selecting large-scale examples rich in formal knowledge, and another by generating informal-to-formal reasoning trajectories guided by expert-designed templates. We then apply SFT and RLVR with these datasets to further fuse and refine the two abilities. The resulting 7B and 32B models exhibit both comprehensive formal knowledge and strong informal-to-formal reasoning. Notably, StepFun-Formalizer-32B achieves SOTA BEq@1 scores of 40.5% on FormalMATH-Lite and 26.7% on ProverBench, surpassing all prior general-purpose and specialized models.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext a6655049-b43a-47cd-96bd-45e481eac579Cited by top-tier papers7
- Aria: an Agent for Retrieval and Iterative Auto-Formalization via Dependency GraphHanyu Wang, Ruohan Xie, Yutong Wang, Guoxiong Gao et al.ICLR 2026 · 21 citations
- ProofFlow: A Dependency Graph Approach to Faithful Proof AutoformalizationRafael Cabral, Tuan Manh Do, Xuejun Yu, Wai Ming Tai et al.ICLR 2026 · 21 citations
- ProofBridge: Auto-Formalization of Natural Language Proofs in Lean via Joint EmbeddingsPrithwish Jana, Kaan Kale, Ahmet Ege Tanriverdi, Cruise Song et al.ICLR 2026 · 19 citations
- ASSESS: A Semantic and Structural Evaluation Framework for Statement SimilarityXiaoyang Liu, Tao Zhu, Zineng Dong, Yuntian Liu et al.ICLR 2026 · 9 citations
- DRIFT: Decompose, Retrieve, Illustrate, then Formalize TheoremsMeiru Zhang, Philipp Borchert, Milan Gritta, Gerasimos LampourasICLR 2026 · 8 citations
Builds on11
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code GenerationJiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming ZhangNeurIPS 2023 · 2,317 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Autoformalization with Large Language ModelsYuhuai Wu, Albert Qiaochu Jiang, Wenda Li, Markus N. Rabe et al.NeurIPS 2022 · 364 citations
- miniF2F: a cross-system benchmark for formal Olympiad-level mathematicsKunhao Zheng, Jesse Michael Han, Stanislas PoluICLR 2022 · 342 citations
Related papers
- Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic ConsistencyZenan Li, Yifan Wu, Zhaoyu Li, Xinming Wei et al.NeurIPS 2024 · 48 citations
- ReForm: Reflective Autoformalization with Prospective Bounded Sequence OptimizationGuoxin Chen, Jing Wu, Xinjie Chen, Xin Zhao et al.ICLR 2026 · 22 citations
- Mathesis: Towards Formal Theorem Proving from Natural LanguagesXuejun Yu, Jianyuan Zhong, Zijin Feng, Pengyi Zhai et al.ICLR 2026 · 15 citations
- miniF2F-Lean Revisited: Reviewing Limitations and Charting a Path ForwardAzim Ospanov, Farzan Farnia, Roozbeh MohitNeurIPS 2025 · 14 citations
- Theorem Prover as a Judge for Synthetic Data GenerationJoshua Ong Jun Leang, Giwon Hong, Wenda Li, Shay B. CohenACL 2025
