Lune

EMNLP Findings2025

What Makes for Good Image Captions?

Delong Chen, Samuel Cahyawijaya, Etsuko Ishii, Ho Shu Chan, Yejin Bang, Pascale Fung

Abstract

This paper establishes a formal informationtheoretic framework for image captioning, conceptualizing captions as compressed linguistic representations that selectively encode semantic units in images. Our framework posits that good image captions should balance three key aspects: informationally sufficient, minimally redundant, and readily comprehensible by humans. By formulating these aspects as quantitative measures with adjustable weights, our framework provides a flexible foundation for analyzing and optimizing image captioning systems across diverse task requirements. To demonstrate its applicability, we introduce the Pyramid of Captions (PoCa) method, which generates enriched captions by integrating local and global visual information. We present both theoretical proof that PoCa improves caption quality under certain assumptions, and empirical validation of its effectiveness across various image captioning models and datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TV2TV: A Unified Framework for Interleaved Language and Video Generation cites Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception
  • TV2TV: A Unified Framework for Interleaved Language and Video Generation cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception cites Visual Instruction Tuning
  • Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
  • Chain-of-Talkers (CoTalk): Fast Human Annotation of Dense Image Captions cites RemoteSAM: Towards Segment Anything for Earth Observation
  • Chain-of-Talkers (CoTalk): Fast Human Annotation of Dense Image Captions cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. RemoteSAM: Towards Segment Anything for Earth ObservationLiang Yao, Fan Liu, Delong Chen, Chuanyi Zhang et al.ACM MM 2025 · 28 citations
  2. TV2TV: A Unified Framework for Interleaved Language and Video GenerationXiaochuang Han, Youssef Emad, Melissa Hall, John Nguyen et al.CVPR 2026 · 3 citations
  3. Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local PerceptionRuotian Peng, Haiying He, Yake Wei, Yandong Wen et al.CVPR 2025
  4. Chain-of-Talkers (CoTalk): Fast Human Annotation of Dense Image CaptionsYijun Shen, Delong Chen, Fan Liu, Xingyu Wang et al.EMNLP 2025

Builds on28

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  3. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  4. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
  5. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations

Related papers

  1. Image Captioning with Context-Aware Auxiliary GuidanceZeliang Song, Xiaofei Zhou, Zhendong Mao, Jianlong TanAAAI 2021 · 36 citations
  2. Image Quality Caption with Attentive and Recurrent Semantic Attractor NetworkWen Yang, Jinjian Wu, Leida Li, Weisheng Dong et al.ACM MM 2021 · 6 citations
  3. Journalistic Guidelines Aware News Image CaptioningXuewen Yang, Svebor Karaman, Joel R. Tetreault, Alejandro JaimesEMNLP 2021
  4. Bridging the Gap between Vision and Language Domains for Improved Image CaptioningFenglin Liu, Xian Wu, Shen Ge, Xiaoyu Zhang et al.ACM MM 2020 · 13 citations
  5. CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image CaptioningZhijiang Tang, Linhua Wang, Jiaxin Qi, Weihao Jiang et al.CVPR 2026 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines