Lune

EMNLP Findings2025

A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks

Haorui Yu, Ramon Ruiz-Dolz, Qiufeng Yi

Abstract

This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM's ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https: //github.com/yha9806/VULCA-EMNLP2025.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Ferret: Refer and Ground Anything Anywhere at Any Granularity cites Visual Instruction Tuning
  • GalleryGPT: Analyzing Paintings with Large Multimodal Models cites Visual Instruction Tuning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective ResponsesChongyuan Dai, Yaling Shen, Zihan Gao, Jia Li et al.ACL 2026 · 4 citations

Builds on8

  1. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  2. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  3. Ferret: Refer and Ground Anything Anywhere at Any GranularityHaoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du et al.ICLR 2024 · 515 citations
  4. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language ModelsNoah Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu et al.ACL Findings 2024 · 120 citations
  5. GalleryGPT: Analyzing Paintings with Large Multimodal ModelsYi Bin, Wenhao Shi, Yujuan Ding, Zhiqiang Hu et al.ACM MM 2024 · 35 citations

Related papers

  1. Capabilities and Evaluation Biases of Large Language Models in Classical Chinese Poetry Generation: A Case Study on Tang PoetryBolei Ma, Yina Yao, Anna-Carolina HaenschACL Findings 2026
  2. Can MLLMs Understand the Deep Implication Behind Chinese Images?Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du et al.ACL 2025
  3. VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial UnderstandingZhaowei Liu, Xin Guo, Haotian Xia, Lingfeng Zeng et al.EMNLP 2025
  4. Poller: Are LLMs Suitable for Evaluating Poetry Understanding Task?Shanshan Wang, Derek F. Wong, Jingming Yao, Lidia S. ChaoACL Findings 2026
  5. AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language ModelsYuhang Wu, Wenmeng Yu, Yean Cheng, Yan Wang et al.ACL 2025 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines