Unstitching the Chimera: Frame-Level Risk and Train-Free Mitigation for Video Hallucination
Songyuan Yang, Guijian Tang, Kun Hu, Haotian Wang, Shixuan Liu, Wenjing Yang, Long Lan, Huibin Tan
Abstract
Hallucination limits the reliability of multimodal large language models (MLLMs), and it is particularly damaging in video where errors manifest as distorted narratives rather than single-frame mistakes. We introduce a frame-first study of Chimera Hallucination: model stitches visual segments that exist in space and time but do not belong to the same event chain, producing a spurious continuous story. We introduce CH-Risk, a single-forward, reference-free risk estimate tailored to this failure mode. CH-Risk combines two complementary signals: SegCoverage@\alpha (\mathrm{SCR}@\alpha\) measures how many event segments are needed to cover most text-to-frame support, exposing long-range stitching; Alignment with Early Temporal Pathway (AETP) measures rank consistency between support and the temporal pathway formed in early–middle layers, exposing stage mismatch. To turn risk into correction, we further propose CH-M(itigation), a train-free two-stage intervention. Segment-aligned Stage-Aligned Frame Routing (sSAFR) re-weights frames before the mid-layer softmax to route attention toward a small set of pathway-aligned segments. Residual Token Calibration (RTC) then stabilizes token usage within selected segments. Extensive experiments across 9 benchmarks and 6 VideoLLMs show that CH-Risk can predict Chimera and that CH-M consistently reduce hallucination and improves task accuracy with negligible overhead (sub-5% latency, sub-2.5% memory, $$1% FLOPs).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 637dfa6e-198f-435a-9a08-1715db6295bfBuilds on21
- Video-LLaVA: Learning United Visual Representation by Alignment Before ProjectionBin Lin, Yang Ye, Bin Zhu, Jiaxi Cui et al.EMNLP 2024 · 231 citations
- Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language ModelsMingrui Wu, Jiayi Ji, Oucheng Huang, Jiale Li et al.ICML 2024 · 32 citations
- LVBench: An Extreme Long Video Understanding BenchmarkWeihan Wang, Zehai He, Wenyi Hong, Yean Cheng et al.ICCV 2025 · 28 citations
- Vista-llama: Reducing Hallucination in Video Language Models via Equal Distance to Visual TokensFan Ma, Xiaojie Jin, Heng Wang, Yuchen Xian et al.CVPR 2024 · 11 citations
- Map the Flow: Revealing Hidden Pathways of Information in VideoLLMsMinji Kim, Taekyung Kim, Bohyung HanICLR 2026 · 8 citations
Related papers
- SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention CollapseYiming Sun, Mi Zhang, Feifei Li, Geng Hong et al.AAAI 2026 · 5 citations
- One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs HallucinationZhan Fa, Yue Duan, Jian Zhang, Lei Qi et al.CVPR 2026 · 2 citations
- Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language ModelsChengsheng Zhang, Chenghao Sun, Xinyan Jiang, Wei Li et al.CVPR 2026 · 2 citations
- Grounding or Guessing? Visual Signals for Detecting Hallucinations in Sign Language TranslationYasser Hamidullah, Koel Dutta Chowdhury, Yusser Al Ghussin, Shakib Yazdani et al.ICLR 2026 · 3 citations
- Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal SteeringShuliang Liu, Songbo Yang, Dong Fang, Sihang Jia et al.ACL 2026 · 9 citations
