Proof-of-Perception: Certified Tool-Using Multimodal Reasoning with Compositional Conformal Guarantees
Arya Fayyazi, Haleh Akrami
Abstract
We present Proof-of-Perception (PoP), a tool-using framework that casts multimodal reasoning as an executable graph with explicit reliability guarantees. Each perception or logic node outputs a conformal set , yielding calibrated, stepwise uncertainty; a lightweight controller uses these certificates to allocate compute under a budget—expanding with extra tool calls only when needed and stopping early otherwise. This grounds answers in verifiable evidence, reduces error compounding and hallucinations, and enables principled accuracy–compute trade-offs. Across document, chart, and multi-image QA benchmarks, PoP improves performance and reliability over strong chain-of-thought, ReAct-style, and program-of-thought baselines while using computation more efficiently.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 154ff714-aac7-4739-8f30-85b2e9ad8c61Cited by top-tier papers1
Ask how each one uses itBuilds on6
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
- Pix2Struct: Screenshot Parsing as Pretraining for Visual Language UnderstandingKenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu et al.ICML 2023 · 426 citations
- MultiDoc2Dial: Modeling Dialogues Grounded in Multiple DocumentsSong Feng, Siva Sankalp Patel, Hui Wan, Sachindra JoshiEMNLP 2021 · 42 citations
- ReAct: Synergizing Reasoning and Acting in Language ModelsShunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du et al.ICLR 2023
Related papers
- Perceptual-Evidence Anchored Reinforced Learning for Multimodal ReasoningChi Zhang, Haibo Qiu, Qiming Zhang, Yufei Xu et al.CVPR 2026 · 22 citations
- See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMsYongchang Zhang, Xianzheng Ma, Tianyi Liu, Guangquan Zhou et al.CVPR 2026 · 2 citations
- More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning ModelsZhongxing Xu, Chengzhi Liu, Qingyue Wei, Juncheng Wu et al.NeurIPS 2025 · 103 citations
- Improving Vision-language Models with Perception-centric Process Reward ModelsYingqian Min, Kun Zhou, Yifan Li, Yuhuan Wu et al.CVPR 2026 · 3 citations
- Inference-Time Conformal Reasoning with Valid Factuality Control for Large Language ModelsTing Wang, Yuanjie Shi, Yan Yan, Huan ZhangICML 2026
