Lune

ACL Findings2026

AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce

Biao Zhang, Lixin Chen, Bin Zhang, Zongwei Wang, Tong Liu, Bo Zheng

Abstract

Multimodal representation is crucial for Ecommerce tasks such as identical product retrieval. Large representation models (e.g., VLM2Vec) demonstrate strong multimodal understanding capabilities, yet they struggle with fine-grained semantic comprehension, which is essential for distinguishing highly similar items. To address this, we propose Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning (AFMRL), which defines product fine-grained understanding as an attribute generation task. It leverages the generative power of Multimodal Large Language Models (MLLMs) to extract key attributes from product images and text, and enhances representation learning through a two-stage training framework: 1) Attribute-Guided Contrastive Learning (AGCL), where the key attributes generated by the MLLM are used in the imagetext contrastive learning training process to identify hard samples and filter out noisy false negatives. 2) Retrieval-aware Attribute Reinforcement (RAR), where the improved retrieval performance of the representation model postattribute integration serves as a reward signal to enhance MLLM's attribute generation during multimodal fine-tuning. Extensive experiments on large-scale E-commerce datasets demonstrate that our method achieves state-of-the-art performance on multiple downstream retrieval tasks, validating the effectiveness of harnessing generative models to advance fine-grained representation learning.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions cites Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
  • Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval cites MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on14

  1. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  2. Align before Fuse: Vision and Language Representation Learning with Momentum DistillationJunnan Li, Ramprasaath R. Selvaraju, Akhilesh Gotmare, Shafiq R. Joty et al.NeurIPS 2021 · 2,985 citations
  3. MagicLens: Self-Supervised Image Retrieval with Open-Ended InstructionsKai Zhang, Yi Luan, Hexiang Hu, Kenton Lee et al.ICML 2024 · 112 citations
  4. M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal PretrainingXiao Dong, Xunlin Zhan, Yangxin Wu, Yunchao Wei et al.CVPR 2022 · 24 citations
  5. Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image RetrievalSiting Li, Xiang Gao, Simon S. DuNeurIPS 2025 · 5 citations

Related papers

  1. CREM: Compression-Driven Representation Enhancement for Multimodal Retrieval and ComprehensionLihao Liu, Biao Yang, Yan Wang, Da Li, Jiangxia Cao, Yuxiao Luo, Xiang Chen, Xiangyu Wu, Wei Yuan, Fan Yang, Guiguang Ding, Tingting Gao, Guorui ZhouCVPR Findings 2026
  2. Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language ModelsHulingxiao He, Geng Li, Zijun Geng, Jinglin Xu et al.ICLR 2025
  3. CTR-Driven Advertising Image Generation with Multimodal Large Language ModelsXingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang et al.WWW 2025 · 15 citations
  4. Scaling Language-centric Omnimodal Representation LearningChenghao Xiao, Hou Pong Chan, Hao Zhang, Weiwen Xu et al.NeurIPS 2025 · 25 citations
  5. Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision EncoderSiting Li, Pang Wei Koh, Simon Shaolei DuACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines