VERIFY: A Novel Multi-Domain Dataset Grounding LTL in Contextual Natural Language via Provable Intermediate Logic
Paapa Quansah, Pablo Rivas, Ernest Bonnah
Abstract
Bridging the gap between the formal precision of system specifications and the nuances of human language is critical for reliable engineering, robotics, and AI safety, but it remains a major bottleneck. Prior efforts in grounding formal logic remain fragmented, resulting in datasets that are very small-scale ( 2-5k examples), domain-specific, or translate logic into overly technical forms rather than context-rich natural language (NL). Thus, failing to adequately bridge formal methods and practical NLP. To address this gap, we introduce VERIFY, the first large-scale dataset meticulously designed to unify these elements. This dataset contains more than 200k+ rigorously generated triplets, each comprising a Linear Temporal Logic (LTL) formula, a structured, human-readable 'Intermediate Technical Language' (ITL) representation designed as a bridge between logic and text, and a domain-specific NL description contextualized across 13 diverse domains. VERIFY's construction pipeline ensures high fidelity: LTL formulas are enumerated and verified via model checking, mapped to the novel ITL representation using a provably complete formal grammar, and then translated into context-aware NL via LLM-driven generation. We guarantee data quality through extensive validation protocols, i.e., manual expert verification of 10,000 diverse samples. Furthermore, automated semantic consistency checks judged by Llama 3.3 confirmed an estimated >97% semantic correctness. From the initial experiments, we demonstrate VERIFY's scalability, logical complexity, and contextual diversity, significantly challenging standard models such as T5 and Llama 3.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext f0afcbac-fca2-44b0-acd5-76e9e36fdad1Builds on1
Related papers
- Bridging Natural Language and Formal Specification-Automated Translation of Software Requirements to LTL via Hierarchical Semantics Decomposition Using LLMsZhi Ma, Cheng Wen, Zhexin Su, Xiao Liang et al.ASE 2025 · 3 citations
- ADARULE: LLM-Driven Natural Language to LTL Conversion via Pattern-Adaptive Rule InductionJiayi Hu, Jingling Sun, Chong Wang, Yihao Huang et al.ICSE 2026
- Automating Requirements Formalization: Using LLMs and Low-Complexity Distinguishing Traces for Semantic ValidationDaniel Mendoza, Anastasia Mavridou, Andreas Katis, Caroline TrippelICSE 2026
- AnalogVerifier: A Neuro-Symbolic Framework for Analog Circuit VerificationYanfang Liu, Mingjun Wang, Peng XU, Rongliang Fu et al.ICML 2026
- On the Limit of Language Models as Planning FormalizersCassie Huang, Li ZhangACL 2025
