STARS: Static Analysis-Guided Assertion Synthesis using Large Language Models
Jialun Cao, Haoyu Wang, Haoran Yan, Ming Wen, Michael Pradel
摘要
Automated unit test generation promises to reduce the cost of software quality assurance, and hence, is attracting attention from both academia and industry. Yet, generating assertions that are executable, meaningful to developers, and able to catch faults remains an unsolved challenge. Existing approaches either randomly enumerate assertions that are plausible based on static program analysis without considering whether they naturally fit the test prefix or query an LLM to generate assertions based on local context only, such as the test prefix and the focal method. However, we observe that local context alone is insufficient for LLMs to generate high-quality assertions because many desirable assertions are built from components that are almost impossible to guess for an LLM, such as sequences of multiple method calls. This paper presents STARS, a novel test assertion generation technique that combines the benefits of static program analysis and LLM-based synthesis. The key idea is to first gather a set of assertion components based on static program analysis and to then combine, concretize, prioritize, and improve them with an LLM. The resulting assertions go beyond what an LLM alone could realistically guess based on the test prefix and focal method, and they naturally fit the given test case. Empirical results show that STARS consistently outperforms the state-of-the-art baseline in five evaluation metrics. STARS achieves an exact-match rate of at most 83.4% using GPT-5.4. Compared with the baseline, STARS’s mutation score nearly triples that of the baseline (10.97% vs. 3.97%), approaching that of developer-written assertions, while consuming 24.75% fewer tokens and 30.6% fewer LLM queries.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- Towards More Realistic Assertion Generation under Mixed-Assertion ScenarioHongyan Li, Kunpeng E, Weifeng Sun, Quanjun Zhang 等ISSTA 2026
- An Empirical Study on Focal Methods in Deep-Learning-Based Approaches for Assertion GenerationYibo He, Jiaming Huang, Hao Yu, Tao XieFSE 2024 · 被引用 8 次
- Evaluating and Improving ChatGPT for Unit Test GenerationZhiqiang Yuan, Mingwei Liu, Shiji Ding, Kaixin Wang 等FSE 2024 · 被引用 89 次
- From Natural Language to Executable Properties for Property-Based Testing of Mobile Apps (Experience Paper)Yiheng Xiong, Ting Su, Jingling Sun, Jue Wang 等ISSTA 2026
- On learning meaningful assert statements for unit test casesCody Watson, Michele Tufano, Kevin Moran, Gabriele Bavota 等ICSE 2020 · 被引用 96 次
