EMNLP Findings2024
FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking
Zhuoer Wang, Leonardo F. R. Ribeiro, Alexandros Papangelis, Rohan Mukherjee, Tzu-Yen Wang, Xinyan Zhao, Arijit Biswas, James Caverlee, Angeliki Metallinou
Abstract
API call generation is the cornerstone of large language models' tool-using ability that provides access to the larger world. However, existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user's request. To address these limitations, we propose an output-side optimization approach called FANTASE. Two of the unique contributions of FANTASE are its State-Tracked Constrained Decoding (SCD) and Reranking components. SCD dynamically incorporates appropriate API constraints in the form of Token Search Trie for efficient and guaranteed generation faithfulness with respect to the API documentation. The Reranking component efficiently brings in the supervised signal by leveraging a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model. We demonstrate the superior performance of FAN-TASE in API call generation accuracy, inference efficiency, and context efficiency with DSTC8 and API Bank datasets. Our code
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- PackMonitor: Towards Zero Package Hallucinations via Decoding-Time Monitoring cites Lookahead-Then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Cited by top-tier papers2
- PackMonitor: Towards Zero Package Hallucinations via Decoding-Time MonitoringXiting Liu, Yuetong Liu, Yitong Zhang, Jia Li et al.ISSTA 2026
- Lookahead-Then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free GrammarsYitong Zhang, Yongmin Li, Yuetong Liu, Jia Li et al.ISSTA 2026
Builds on14
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
- Plug and Play Language Models: A Simple Approach to Controlled Text GenerationSumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung et al.ICLR 2020 · 1,166 citations
Related papers
- Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic ParsingShufan Wang, Sébastien Jean, Sailik Sengupta, James Gung et al.EMNLP Findings 2023 · 5 citations
- The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained DecodingAvinash Reddy, Thayne Walker, Jaime Ide, Amrit Singh BediICML 2026 · 6 citations
- Earley-Driven Dynamic Pruning for Efficient Structured DecodingXintong Sun, Chi Wei, Minghao Tian, Shiwen NiICML 2025
- ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function CallingJianghao Lin, Yuanyuan Shi, Xin Peng, Renjie Ding et al.ACL 2026 · 3 citations
- Grammar-Aligned DecodingKanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova et al.NeurIPS 2024 · 73 citations
