Lune

CVPR Findings2026

Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination

Xinzhuo Li, Adheesh Juvekar, Jiaxun Zhang, Xingyou Liu, Muntasir Wahed, Kiet A. Nguyen, Yifan Shen, Tianjiao Yu, Ismini Lourentzou

Abstract

tion hallucinations using controlled visual counterfactuals, alongside new evaluation metrics that measure hallucination severity and disentangle vision-and language-driven failure modes. We further introduce RobustSeg, a segmentation VLM trained with counterfactual finetuning (CFT) to learn when to segment and when to abstain. Experimental results confirm RobustSeg reduces hallucinations by 30%, while improving segmentation performance on FP-RefCOCO(+/g). PLAN Lab https://plan-lab.github.io/hallusegbench

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • 3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding cites Visual Instruction Tuning
  • 3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • 3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. 3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive DecodingMakanjuola Adekunmi Ogunleye, Eman Abdelrahman, Ismini LourentzouCVPR 2026 · 3 citations

Builds on34

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
  3. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  4. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
  5. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations

Related papers

  1. CF-VLM: CounterFactual Vision-Language Fine-tuningJusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang et al.NeurIPS 2025 · 71 citations
  2. Leveraging Hallucinations to Reduce Manual Prompt Dependency in Promptable SegmentationJian Hu, Jiayi Lin, Junchi Yan, Shaogang GongNeurIPS 2024 · 39 citations
  3. From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language ModelsYuying Shang, Xinyi Zeng, Yutao Zhu, Xiao Yang et al.ACM MM 2025 · 5 citations
  4. Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal InterventionZhe Xu, Zhicai Wang, Junkang Wu, Jinda Lu et al.AAAI 2026
  5. Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive DecodingSicong Leng, Hang Zhang, Guanzheng Chen, Xin Li et al.CVPR 2024
Dusk over the sea between two cliffs drawn in fine vertical lines