Lune

EMNLP Findings2025

HMCL: Task-Optimal Text Representation Adaptation through Hierarchical Contrastive Learning

Zhenyi Wang, Yapeng Jia, Haiyan Ning, Peng Wang, Dan Wang, Yitao Cao

Abstract

As general large language models continue to advance, their real-world adaptation through effective fine-tuning remains a significant challenge. We introduce Hierarchical Multilevel Contrastive Learning (HMCL), a new contrastive learning framework that improves taskspecific text representation for general models. HMCL integrates 3-level semantic differentiation (positive, weak-positive, and negative) and unifies contrastive learning, pair classification, and ranking objectives into a cohesive optimization strategy. HMCL demonstrates exceptional results across multi-domain and multilingual benchmarks, including text similarity, retrieval, reranking and Retrieval-Augmented Generation (RAG) tasks. It outperforms top unsupervised methods and supervised fine-tuning approaches while maintaining broad compatibility with architectures ranging from BERT to Qwen, 330M to 7B. In real-world merchant consultation scenarios, HMCL shows a 0.70-6.24 point improvement over original fine-tuning methods in large-scale base models. This establishes HMCL as a versatile solution that bridges the gap between general-purpose models and specialized industrial applications.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites A Simple Framework for Contrastive Learning of Visual Representations
  • M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation cites SimCSE: Simple Contrastive Learning of Sentence Embeddings
  • SimCSE: Simple Contrastive Learning of Sentence Embeddings cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on11

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  3. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  4. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationJianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo et al.ACL Findings 2024 · 762 citations
  5. MIND: A Large-scale Dataset for News RecommendationFangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu et al.ACL 2020 · 454 citations

Related papers

  1. Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text ClassificationZihan Wang, Peiyi Wang, Lianzhe Huang, Xin Sun et al.ACL 2022 · 157 citations
  2. CREM: Compression-Driven Representation Enhancement for Multimodal Retrieval and ComprehensionLihao Liu, Biao Yang, Yan Wang, Da Li, Jiangxia Cao, Yuxiao Luo, Xiang Chen, Xiangyu Wu, Wei Yuan, Fan Yang, Guiguang Ding, Tingting Gao, Guorui ZhouCVPR Findings 2026
  3. AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerceBiao Zhang, Lixin Chen, Bin Zhang, Zongwei Wang et al.ACL Findings 2026
  4. Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM RerankingXin Zhang, Ziqi Dai, Mingxin Li, Yanzhao Zhang et al.ICLR 2026 · 8 citations
  5. Scalable Tabular Hierarchical Metadata Classification in Heterogeneous Structured Large-Scale Datasets Using Contrastive LearningBhimesh Kandibedala, Gyanendra Shrestha, Anna Pyayt, Todor Ivanov et al.ICDE 2025
Dusk over the sea between two cliffs drawn in fine vertical lines