Revisiting Visual Corruptions in LVLMs: A Shape-Texture Perspective on Model Failures
Xinkuan Qiu, Meina Kan, Zhenliang He, Yongbin Zhou, Shiguang Shan
摘要
Large vision-language models (LVLMs) are highly vulnerable to visual corruptions, compromising their reliability and limiting real-world deployment. Prior work has attributed this degradation primarily to insufficient visual grounding and overreliance on language priors. However, these explanations often overlook the heterogeneous nature of corruptions, which perturb model perception in fundamentally different ways. We revisit this problem from a corruption-centric view and show that diverse corruptions can be organized along two complementary perceptual dimensions-shape and texture-which induce distinct failure modes. To address them, we propose Shape-Texture Dual-Path Contrastive Decoding (ST-CD), a training-free inference framework that constructs complementary contrastive views to diagnose and correct shape-and textureinduced biases through adaptive fusion. Experiments across multiple LVLMs and robustness benchmarks demonstrate that ST-CD consistently improves robustness under heterogeneous corruptions, suggesting that leveraging the complementarity between shape and texture provides a general and effective principle for robust multimodal models.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper8
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- The Origins and Prevalence of Texture Bias in Convolutional Neural NetworksKatherine L. Hermann, Ting Chen, Simon KornblithNeurIPS 2020 · 被引用 369 次
- Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang 等EMNLP 2023 · 被引用 344 次
- Informative Dropout for Robust Representation Learning: A Shape-bias PerspectiveBaifeng Shi, Dinghuai Zhang, Qi Dai, Zhanxing Zhu 等ICML 2020 · 被引用 122 次
- Contrastive Decoding: Open-ended Text Generation as OptimizationXiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang 等ACL 2023 · 被引用 78 次
相关 Paper
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive DecodingSicong Leng, Hang Zhang, Guanzheng Chen, Xin Li 等CVPR 2024
- Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language ModelsBingqian Liu, Fu Zhang, Guoqing Chen, Jingwei ChengEMNLP 2025
- Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference FrameworkKaihua Tang, Jiaxin Qi, Jinli Ou, Yuhua Zheng 等CVPR 2026 · 被引用 1 次
- Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language ModelsWei Chen, Xin Yan, Bin Wen, Fan Yang 等NeurIPS 2025 · 被引用 4 次
- Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language ModelsEun Woo Im, Muhammad Kashif Ali, Vivek GuptaICLR 2026 · 被引用 1 次
