CEBC: Conformal Evidence-Bounded Control for Low-Hallucination Vision-Language Generation

Hallucinated object mentions remain a persistent failure mode of vision-language models (VLMs) across generation tasks such as image captioning and visual question answering: outputs may be fluent, yet include entities not supported by visual evidence. Existing mitigation approaches often reduce hallucinations at the cost of degraded generation quality or require expensive retraining and taskspecific supervision. We introduce CEBC, a lightweight, training-free framework for lowhallucination vision-language generation based on conformal evidence-bounded minimal editing. CEBC first produces a strong base output (via greedy decoding or best-of-K sampling), then applies an evidence-bounded editing step that minimally revises or suppresses unsupported object mentions using constraints derived from an external visual detector. Crucially, the evidence threshold is conformally calibrated on a small held-out set via quantiles of detector confidence scores, enabling explicit and controllable hallucination risk at test time. To balance factuality and informativeness, we further introduce a risk-first, qualityaware selection rule that prioritizes evidenceconsistent generations while regularizing unnecessary length or lexical drift. Extensive experiments on MS-COCO and GQA for image captioning, and POPE for VQA evaluation across multiple VLMs demonstrate that CEBC consistently reduces hallucination rates (CHAIR S , CHAIR I , POPE) while maintaining or improving standard generation quality metrics (CIDEr, BLEU, CLIPScore). CEBC establishes a stronger factuality-quality Pareto frontier without any additional model training or access to paired supervision beyond an offthe-shelf detector.