Lune

ACL Findings2026

Long Story Short: Disentangling Compositionality and Long-Caption Understanding in Contrastive VLMs

Israfel Salazar, Desmond Elliott, Yova Kementchedjhieva

Abstract

Contrastive vision-language models (VLMs) have made significant progress in binding visual and textual information, yet understanding long, compositional captions remains an open challenge. While these capabilities are often assumed to be closely related, the conditions under which they reinforce each other remain unclear. In this paper, we empirically analyze when compositional reasoning and longcaption understanding transfer across tasks, and when this relationship fails. Through controlled experiments across diverse training objectives, datasets, and architectural designs, we find a bidirectional but sensitive relationship between the two capabilities. Models trained on poorly grounded captions or with limited parameter updates fail to generalize, while high-quality long-caption data with strong visual grounding promotes both capabilities simultaneously. We further show that architectural choices aimed at preserving general alignment, such as frozen positional embeddings, can inadvertently limit compositional learning. Our analysis provides actionable guidelines for data selection and model design to improve VLM generalization.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives cites Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
  • TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives cites Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
  • Sigmoid Loss for Language Image Pre-Training cites Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on14

  1. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen et al.ICML 2021 · 5,401 citations
  2. Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 2,932 citations
  3. PaLI: A Jointly-Scaled Multilingual Language-Image ModelXi Chen, Xiao Wang, Soravit Changpinyo, A. J. Piergiovanni et al.ICLR 2023 · 194 citations
  4. Winoground: Probing Vision and Language Models for Visio-Linguistic CompositionalityTristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh et al.CVPR 2022 · 179 citations
  5. TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language NegativesMaitreya Patel, Abhiram Kusumba, Sheng Cheng, Changhoon Kim et al.NeurIPS 2024 · 73 citations

Related papers

  1. Unveiling the Compositional Ability Gap in Vision-Language Reasoning ModelTianle Li, Jihai Zhang, Yongming Rao, Yu ChengNeurIPS 2025 · 17 citations
  2. Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision EncoderSiting Li, Pang Wei Koh, Simon Shaolei DuACL 2025
  3. Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional UnderstandingLe Zhang, Rabiul Awal, Aishwarya AgrawalCVPR 2024 · 7 citations
  4. CF-VLM: CounterFactual Vision-Language Fine-tuningJusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang et al.NeurIPS 2025 · 71 citations
  5. No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive ModelsHai X. Pham, David T. Hoffmann, Ricardo Guerrero, Brais MartínezCVPR 2026 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines